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EXECUTIVE  SUMMARY 

1.  A  Human  Factors  study  was  conducted  on  B-1B  Blocks  D,  E,  and  F.  The  1st  study 
objective  was  to  evaluate  B-1B  Blocks  D,  E,  and  F  crew  workload  and  situational 
awareness.  The  2nd  objective  was  to  establish  a  measurable  crew  performance 
baseline  for  future  B-1B  Block  upgrades. 

2.  These  objectives  were  studied  in  the  B-1B  Engineering  Research  Simulator  (ERS)  in 
a  simulated  full  mission  environment  using  eight  B-1B  crews.  After  training,  the 
crews  flew  two  missions.  Workload  and  situational  awareness  were  measured,  a 
questionnaire  was  given,  and  all  missions  were  video  taped. 

3.  The  main  findings  of  the  study  are  that: 

a.  Control  and  display  improvements  to  the  B-1B  Block  E  Launch  Acceptability 
Region  (LAR)  display  (EB  Page)  are  recommended  to  reduce  the  Offensive 
System  Operator’s  (OSO)  workload 

b.  Improved  targeting  information  is  needed  on  the  Target  Summary  display  (E 
Page). 

c.  Improved  hung  store  training  is  needed  for  OSOs. 

d.  A  Digital  Bull,  Bra,  and  Compass  Rose  should  be  added  as  soon  as  possible  to 
the  DSO’s  Tactical  Situation  Display  (TSD). 

e.  Defensive  System  Operator  (DSO)  threat  re-identification  capability  should  be 
added. 

f.  The  Inter-Leavened  Search  and  Track  (ILST)  radar  mode  is  mechanized  correctly 
and  is  useful. 

g.  An  ALR-56M  radar  warning  receiver  Repeater  should  be  added  to  the  front 
station. 

h.  Weapon  capability  training  for  Joint  Direct  Attack  Munitions  (JDAMs)  and  Wind 
Corrected  Munition  Dispensers  (WCMDs)  is  needed. 

i.  A  Data  Link,  Heads  Up  Display  (HUD),  and  Color  Heads  Down  Displays  are  the 
top  three  rated  items  that  should  be  included  in  future  Block  Upgrades  for  the 
pilot  and  copilot. 

j.  A  Data  Link,  Color  Heads  Down  Displays,  and  Moving  Map  are  the  top  three 
rated  items  that  should  be  included  in  future  Block  Upgrades  for  the  Weapon 
System  Operators  (WSOs). 
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1.0  INTRODUCTION 

The  B-1B  System  Program  Office  Engineering  Division  (ASC/YDE)  in  conjunction  with 
the  Crew  Station  Evaluation  Facility  (CSEF)  of  the  Engineering  Directorate  of 
Aeronautical  Systems  Center  (ASC/ENFC)  has  conducted  a  series  of  applied  research 
studies  in  human  factors  engineering  and  crew  station  design.  They  have  used  the  B-1B 
ERS  as  a  major  tool  in  these  endeavors.  Some  examples  of  past  efforts  include  Defensive 
System  Upgrade  Program  (DSUP)  studies.  Conventional  Mission  Upgrade  Program 
(CMUP)  studies.  Block  Upgrade  studies,  Crew  Station  Working  Group  (CSWG)  support, 
and  a  Link  16  study.  Government  engineers  have  worked  closely  with  the  contractors, 
Boeing  (B-1B)  and  Raytheon  Training  Inc.  (CSEF),  to  support  these  human  factors 
efforts.  The  products  of  these  studies  have  been  folded  back  into  the  B-1B  system  to 
make  it  a  more  useable  and  effective  system. 


1.1  STUDY  OB JECTIVE 

This  is  the  first  in  a  series  of  studies  to  look  at  the  effects  of  B-1B  Block  upgrades  on 
crew  workload,  situational  awareness,  and  coordination.  The  objective  of  this  study  is  to 
establish  B-1B  Baseline  crew  performance  and  workload  measures  and  standards  so  that 
future  B-1B  Block  Upgrades  can  be  measured  against  an  established,  quantified  baseline. 
Future  studies  will  build  upon  this  study's  results  to  assess  future  Block  Upgrade's 
impacts  on  crew  performance  and  workload. 

Broadly,  the  study  looked  at: 

The  Block  D  improved  Communication  and  Navigation  Mission  System  (CNMS), 
including  the  Control  Display  Unit  (CDU). 

The  Block  E  unconstrained  JDAM,  WCMD,  the  associated  LAR,  and  ILST. 

The  Block  F  Integrated  Defensive  Electronic  Countermeasures  System  (IDECM), 
the  ALR-56M  Radar  Warning  Receiver  (including  Audio),  and  the  Fiber  Optic 
Towed  Decoy  (FOTD). 

And,  although  not  currently  associated  with  any  Block  upgrade,  the  study  also 
evaluated  Digital  Bull,  Bra,  and  Compass  Rose  on  the  TSD,  and  an  ALR-56M 
Repeater  Scope  on  the  top  center  of  the  Pilot  and  Copilot’s  glare  shield. 
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2.0  METHOD 


2.1  SUBJECTS 

Eight  B-1B  crews  participated  in  the  study.  A  B-1B  crew  has  a  Pilot,  Copilot,  OSO,  and 
DSO.  The  average  pilot/copilot  B-1B  flight  time  was  1212.56  hours  with  a  range  of  150 
to  2600  hours.  The  average  number  of  B-1B  flying  hours  for  WSOs  (i.e.,  either  a  OSO  or 
DSO)  was  966.38  hours  with  a  range  of 250  to  2250  hours.  See  Appendix  1  for  the  Bases 
the  crews  came  from  and  their  participation  schedule.  See  Appendix  2,  B-1B  Baseline 
Study  Questionnaire,  Questions  1  through  8,  for  an  in-depth  look  at  the  crew’s  experience 
profiles. 


2.2  SIMULATION  TEST  BED 

The  CSEF’s  B-1B  ERS  was  used  to  conduct  the  study.  The  CSEF’s  B-1B  ERS  contains 
the  pilot,  co-pilot,  offensive  and  defensive  stations.  The  system  does  not  employ  a 
motion  base.  For  the  Baseline  Study,  the  pilot  and  copilot's  stations  were  in  the  Block  D 
configuration,  the  OSO's  station  was  in  the  Block  E,  and  DSO's  station  was  in  the  Block 
F  configuration. 


2.3  SUBJECT  TRAINING 

Crews  were  given  an  orientation  briefing  covering  the  study’s  purpose,  procedures,  and 
schedule  (see  Appendix  3).  Each  crew  was  briefed  on  B-1B  Blocks  D,  E,  and  F.  Each, 
crew  was  given  B-1B  ERS  familiarization  time  as  well  as  flying  at  least  2  practice 
missions.  See  Appendix  4  for  the  practice  mission’s  details.  See  Appendix  5  for  both  the 
practice  and  the  data  collection  mission’s  briefings. 

The  crews  were  taught  the  Subject  Workload  Assessment  Technique  (SWAT)  and  did  the 
SWAT  card  sort  in  accordance  with  AAMRL-TR-89-023,  Subjective  Workload 
Assessment  Technique  (SWAT):  A  User's  Guide. 

Gary  B.  Reid  -  Armstrong  Aerospace  Medical  Research  Laboratory 
Scott  S.  Potter  and  Jeine  R.  Bressler  -  Systems  Research  Laboratories,  Inc. 

July,  1989 

Interim  Report  for  Period  June  1986-October  1988 


2.4  MISSIONS 

Two  data  collection  missions  were  flown.  See  Appendix  6  for  the  data  collection 
mission’s  details.  The  1st  data  collection  mission  was  a  strike  against  three,  static,  pre¬ 
planned,  “soft”  and  “hard”  target  arrays  using  JDAMs,  both  Mark  84s  and  BLU-109s. 
This  1st  mission  also  had  hung  stores,  a  threat  diversion,  and  a  diversion  to  an  alternative 
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landing  field.  The  1st  data  collection  mission  was  called  “UTTR”.  The  2nd  data 
collection  mission  was  a  strike  against  three  mobile  target  arrays  using  CBU-103 
Combined  Effects  Munitions  (CEMs)  WCMDs.  The  2nd  data  collection  mission  also  had 
a  tanker  rendezvous  and  was  called  “Powder  River”. 

Both  data  collection  missions  were  in  a  high  surface-to-air  threat  air  environment  (see 
Appendix  5,  Slide  5)  with  the  B-1B  assumed  to  be  a  part  of  a  larger  (simulated)  strike 
package.  Both  data  collection  missions  were  direct  adaptations  of  missions  regularly 
flown  by  B-lBs  at  Ellsworth  AFB.  The  data  collection  missions  were  flown  in  a  counter¬ 
balanced  order  (see  Appendix  1). 

2.5  STATION/BLOCK  EMPHASIS 


a.  Pilot/Copilot  -  Block  D 

1)  Communication/Navigation  -  CNMS  CDU 

2)  ARC-210 

b.  OSO  -  Block  E  -  Conventional  Mission  Upgrade  Program  (CMUP) 

1)  JDAM 

3)  WCMD 

4)  ILST 

c.  DSO  -  Block  F  -  Defensive  System  Upgrade  Program  (DSUP) 

1)  ASQ-184  -  Radio  Frequency  Surveillance/Electronics 
Countermeasures  (RFS/ECM)  control  panel,  DSO  Power  Panel 
(PWR),  Integrated  Keyboard  (EKB),  2  Electronic  Display  Units 
(EDUs),  Multi-Function  Display  (MFD),  Graphics  Generator,  DSO 
Track  Handle,  Audio  Changes 

2)  FOTD  (ALR-50)  CDU 

3)  ALR-56M  Radar  Warning  Receiver  (RWR) 

4)  Low  Band  On-Board  Jammer 

Some  out-of-block  features  present  in  the  B-1B  ERS  that  were  taught,  simulated,  and 
discussed  were:  ALR-56M  Display  Repeater  on  the  top  center  of  the  pilot/copilot’s 
glareshield  and  a  DSO’s  TSD  with  a  Digital  Bulls  Eye,  Digital  Bra,  and  a  Compass  Rose. 


2.6  DATA  COLLECTION 

The  following  data  was  collected: 

a.  Questionnaire  -  A  questionnaire  was  developed  to  obtain  the  crew’s 
subjective  ratings  and  comments  on  the  test  conditions.  See  Appendix  2. 
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b.  Workload  -  SWAT.  See  Appendix  7.  SWAT  was  used  to  assess  mental  workload 
by  making  relative  comparisons  of  task  conditions.  SWAT  is  an  accepted  technique 
for  determining  if  a  task  under  one  condition  requires  a  greater  mental  workload  than 
workload  under  another  condition.  SWAT  was  generally  handled  IAW 
"SUBJECTIVE  WORKLOAD  ASSESSMENT  TECHNIQUE  (SWAT):  A  USERS 
GUIDE  (U)"  AAMRL-TR-89-023,  Reid,  Potter,  &Bressler,  July,  189.  SWAT 
Version  3.1  was  used  to  aid  in  the  computer  processing  of  the  SWAT  data.  SWAT  is 
broken  down  into  3  parts:  Time  Load,  Mental  Effort  Load,  and  Psychological  Stress 
Load.  During  training,  crews  accomplished  the  SWAT  Card  Sort.  See  also  Appendix 
3,  Slides  6  -  12.  SWAT  Event  Scores  were  collected  at  4  pre-selected  Mission 
"Freeze"  points. 

c.  Situational  Awareness  (SA).  See  Table  10.  SA  was  generally  handled  IAW  with 
"SITUATIONAL  AWARENESS  GLOBAL  ASSESSMENT  TECHNIQUE 
(SAGAT)  AIR-TO-AIR  TACTICAL  VERSION,  USER'S  GUIDE"  NOR  DOC  89- 
58,  Endsley,  March  7,  1989,  and  "SITUATIONAL  AWARENESS  IN  DYNAMIC 
HUMAN  DECESION  MAKING:  THEORY  AND  MEASUREMENT",  NOR  DOC 
90-49,  Endsley,  May,  1990.  Situational  Awareness  questions  were  randomly  selected 
from  a  list  of  10  possibilities  (see  Appendix  3,  Slides  16, 17,  &  18)  and  were  given  at 
4  pre-selected  Mission  "Freeze"  points  during  a  mission/mission  segment  to  assess 
individual  crew  member's  situational  awareness.  The  possible  questions  were  about 
either  Own  Ship  SA  or  Environment  S A  in  nature.  For  every  crew,  the  same  S A 
Questions  were  asked  at  the  same  freeze  point.  The  crew  SA  responses  were  scored 
as  either  Pass  or  Fail,  that  is,  the  crewmember  either  had  SA  or  didn’t.  Situational 
Awareness  questions  were  asked  just  after  the  taking  of  SWAT  ratings. 

d.  Video  Tape  -  The  crews  were  video  taped  during  all  data  collection  missions. 
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3.0  DATAANLYSIS 


3.1  SWAT  DATA  ANALYSIS 

Statistical  Package  for  the  Social  Sciences  (SPSS),  release  9.0  was  used  to  analyze  the 
SWAT  data.  The  SWAT  data  was  analyzed  as  4  separate  designs.  The  SWAT  data  was 
analyzed  for  the  1)  crew,  2)  pilots,  3)  OSO,  and  4)  DSO.  The  dependent  variable  was 
SWAT,  the  independent  variables  were  mission  and  freeze  point.  The  crew’s  B-1B 
experience  was  treated  as  a  between  subjects  variable.  A  Multivariate  Analysis  of 
Variance  (MANOVA)  was  used  for  all  4  analyses.  Each  design  looked  at  mission  and 
freeze  point.  If  the  SPSS  Multivariate  criterion  tests  were  significant  at  the  alpha  .05 
level,  more  detailed  analyses  were  done. 

An  anomaly  of  the  SWAT  program  prevents  entering  data  for  more  than  30  subjects. 
Since  the  subject  pool  exceeded  30,  the  data  was  divided  into  two  groups:  Pilots  and 
WSOs.  Rescaled  SWAT  values  were  obtained  for  the  pilot  and  WSO  pools.  SWAT 
scores  were  used  for  the  crew  level  (averaged  crew  SWAT  scores),  pilot  level  (averaged 
pilot  and  co-pilot  scores),  and  for  the  WSOs.  For  the  crew  analysis,  the  pilot  and  WSO 
scales  were  averaged.  For  the  other  analyses,  the  pilot  rescaled  values  were  used  for  the 
pilot  analyses,  and  the  WSO  rescaled  values  were  used  for  the  OSO  and  DSO  analyses. 

3.1.1  CREW  ANALYSIS 

For  the  crew  analysis,  each  crew’s  SWAT  scores  were  averaged  together  for  an 
aggregated  SWAT  score  at  each  freeze  point  for  both  missions.  Figures  1  and  2  display 
the  mean  values  for  SWAT  for  mission,  and  mission  and  freeze  point,  respectively. 
Mission  1  has  a  greater  average  SWAT  score  than  Mission  2  (23.97  vs.  19.03)  and 
Mission  1,  freeze  point  4  has  the  greatest  average  SWAT  score  (31.03).  Indices  of 
central  tendency  and  dispersion  are  shown  in  Appendix  8,  Table  1  and  correlation 
coefficients  in  Appendix  8,  Table  5.  There  is  a  significant  correlation  between  SWAT 
and  crew,  and  SWAT  and  freeze  point  (p<  0.01). 

There  were  no  significant  main  effects  for  the  general  MANOVA.  However,  there  was  a 
significant  interaction  for  mission  and  freeze  point,  (p  <  0.05,  Appendix  9,  Table  1). 
Given  the  significant  interaction,  further  analyses  focused  on  the  six  simple  main  effects 
for  mission  and  freeze  point:  four  for  mission  comparing  Mission  1  with  Mission  2  for 
each  level  of  freeze  point,  and  two  comparing  freeze  point  under  each  level  of  mission. 
The  simple  main  effects  for  freeze  point  for  Missions  1  and  2  were  not  significant 
(Appendix  9,  Tables  2-3).  The  simple  main  effects  for  mission  were  not  significant  for 
freeze  points  1,3,  and  4;  however,  they  were  significant  for  freeze  point  2  (p<  0.05, 
Table  1).  Given  significance,  further  analysis  focused  on  the  degree  of  association 
between  the  difference  measure  and  the  original  SWAT  scores  (Eta  Squared).  Eta 
Squared  is  shown  to  be  moderate  and  the  difference  in  SWAT  between  Missions  1  and  2 
is  significant  (p<0.05,  Tables  land  2  respectively). 
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Experience  level,  defined  as  B-1B  experience  hours,  was  analyzed  as  a  between  subjects 
variable  for  the  crew  MANOVA  but  was  shown  to  be  not  significant  (p>0.05). 
Therefore,  it  was  no  longer  considered  in  subsequent  analyses. 


23.97 


16.00- 


MISSION 

Figure  1.  SWAT  MEAM  -  CREW 


31413 


Figure  2.  SWAT  MEAN  BY  MISSION  AND  FREEZE  POINT  -  CREW 
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TABLE  1.  SIMPLE  MAIN  EFFECT  -  MISSION:  TEST  OF  WITHIN  SUBJECTS 
CONTRASTS  FREEZE  POINT  2  CREW 


SWAT 

Source 

MSN 

Type  IV 
Sum  of 
Squares 

df 

Mean 

Square 

F 

Sig. 

Eta 

Squared 

Estimated 

Population 

Eta 

Squared+ 

MSN 

Level  2 
vs. 

Level  1 

271.566 

1 

271.566 

9.544 

.021 

.614 

.555 

Error(MSN) 

A  _ _ i.  x* _ 

Level  2 

vs. 

Level  1 

170.731 

6 

28.455 

j 

+  Adjustment  formula  provided  is  from  Bray,  J.H.  and  Maxwell.  S.  (1990  n  37)  Multivariate  Analytic  jrf 
Variance.  Sage  Publications,  Newbury  Park. 


TABLE  2.  SIMPLE  MAIN  EFFECT  -MISSION:  PAIRWISE  COMPARISION 

FREEZE  POINT  2  CREW 


SWAT 

Bonferroni 

BH 

Mean  Difference  (I- 
J) 

Std. 

Error 

Sig. 

95%  Confidence 
Interval  for  Difference 

(I)  MSN 

Lower  Bound 

Upper  Bound 

1 

2 

6.229 

Bom 

1.295 

11.162 

2 

1 

-6.229 

2.016 

.021 

-11.162 

-1.295 

Based  on  estimated  marginal  means.  The  mean  difference  is  significant  at  the  .05  level. 


3.1.2  PILOT  ANALYSIS 

Figures  3  and  4  show  the  mean  SWAT  values  for  the  pilots  (flight  crew)  for  mission,  and 
mission  and  freeze  point  respectively.  Unlike  the  crew,  OSO  or  DSO,  the  Pilot’s  average 
SWAT  score  was  greatest  for  Mission  2  (19.01  vs.  22.49).  Likewise  the  pilot’s  greatest 
average  SWAT  score  was  for  Mission  2,  freeze  point  3  (33.23).  More  indices  of  central 
tendency  and  dispersion  are  available  in  Appendix  8,  Table  2  and  correlation  coefficients 

in  Appendix  8,  Table  6.  There  was  a  significant  correlation  between  SWAT  and  freeze 
point  (p<0.01). 

The  pilot  SWAT  data  was  analyzed  by  repeated  measures  MANOVA  for  mission  and 
freeze  point.  No  significant  main  effects  or  significant  interactions  were  found 
(Appendix  9,  Table  4).  As  with  the  crew  analysis,  B-1B  experience  was  treated  as  a 
between  subjects  variable  but  was  not  found  to  be  significant  (p>0.05). 
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22.49 


MISSION 


Figure  3.  SWAT  MEAN  -  FLIGHT  CREW 


29-67  2622 


Figure  4.  SWAT  MEAN  BY  MISSION  AND  FREEZE  POINT  -  FLIGHT  CREW 

3.1.3  OSO ANALYSIS 

Figures  5  and  6  depict  the  SWAT  mean  values  for  the  OSO  mission,  and  mission  and 
freeze  point  respectively.  The  greatest  average  SWAT  score  was  for  Mission  1  (40.45  vs. 
20.43)  and  Mission  1,  freeze  point  4  had  the  largest  average  SWAT  value  (51.76). 
Reference  Appendix  8,  Table  3  for  additional  indices  of  central  tendency  and  dispersion 
and  Appendix  8,  Table  7  for  correlation  coefficients.  There  was  a  significant  negative 
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correlation  between  SWAT  and  mission,  and  a  significant  positive  correlation  between 
SWAT  and  freeze  point  (p<0.01). 

The  OSO  SWAT  data  was  analyzed  with  a  repeated  measures  MANOVA  for  mission  and 
freeze  point.  The  MANOVA  was  significant  for  mission  main  effects  and  freeze  point  (p 
<0.05,  Appendix  9,  Table  5).  There  was  no  significant  interaction  (p>0.05).  For  the  test 
of  within  subject  contrasts,  mission  and  freeze  point  (linear  contrasts)  were  significant  (p 
<0.05,  Table  3).  B-1B  experience  was  examined  as  a  between  subjects  variable  but  was 
found  not  to  be  significant  (p<0.05). 

Given  that  the  OSO  MANOVA  was  significant,  a  more  detailed  analysis  using  the  mean 
difference  was  accomplished  and  showed  a  significant  difference  for  mission  (p  <  0.05, 
Table  4).  None  of  the  freeze  point  pair-wise  comparisons  were  significant.  However, 
note  that  the  pair-wise  comparison  for  freeze  point  1  and  4  was  very  close  to  achieving 
significance  (p=0,056,  Table  5).  Sample  Eta  Squared  and  generalized  population  Eta 
Squared  values  are  reported  in  Table  3. 


40.45 


>  2 

MISSION 

Figure  5.  SWAT  MEAN  -  OSO 
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TABLE  4.  PAIRWISE  COMPARISONS:  MISSION  -  OSO 


SWAT 

Bonferroni 

Mean  Difference 
(I-J) 

Std.  Error 

Sig. 

95%  Confidence 
Interval  for 
Difference 

(I)  MSN 

Lower  Bound 

Upper  Bound 

1 

23.000 

4.966 

.019* 

7.195 

38.805 

2 

l 

-23.000 

4.966 

.019* 

-38.805 

-7.195 

♦The  mean  difference  is  significant  at  the  .05  level. 
Based  on  estimated  marginal  means 


TABLE  5.  PAIRWISE  COMPARISON:  MISSION  AND  FREEZE  POINT  -  OSO 


SWAT 

Mean  Difference 

(i-j) 

95%  Confidence 
Interval  for 
Difference a 

(I)  FRZPT 

(J)  FRZPT 

Lower  Bound 

1 

2 

■■SEEE9H1 

7.971 

1.000 

-62.575 

36.775 

3 

■ 

7.051 

.467 

-62.536 

25.336 

4 

-22.063 

-44.970 

.845 

2 

1 

12.900 

■ 

■KHW 

-36.775 

62.575 

3 

7.183 

H 

-50.462 

39.062 

4 

-9.163 

5.663 

■low 

-44.453 

26.128 

3 

1 

18.600 

.467 

-25.336 

62.536 

2 

5.700 

7.183 

1.000 

-39.062 

50.462 

4 

-3.462 

3.692 

1.000 

-26.467 

19.542 

4 

1 

22.063 

3.676 

.056 

-.845 

44.970 

2 

9.163 

5.663 

1.000 

-26.128 

44.453 

3 

3.462 

1.000 

-19.542 

Based  on  Estimated  Marginal  Means 
a.  Adjustment  for  multiple  comparisons:  Bonferroni 


3.1.4  DSO  ANALYSIS 


Figures  7  and  8,  respectively,  depict  the  mean  SWAT  values  for  mission,  and  mission 
and  freeze  point.  Consistent  with  the  crew  and  the  OSO,  the  largest  average  SWAT  score 
was  for  Mission  1  (15.95  vs.  1 1.59).  The  largest  average  SWAT  score  for  mission  and 
freeze  point  was  Mission  2,  freeze  point  3  (22.51).  Not  obvious  from  the  graph,  the 
average  SWAT  score  for  Mission  2,  freeze  point  1  was  zero.  Indices  of  central  tendency 
and  dispersion  are  shown  in  Appendix  8,  Table  4  and  correlation  coefficients  in 
Appendix  8,  Table  8.  There  was  a  significant  correlation  between  SWAT  and  freeze 
point  (p<0.01). 
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The  DSO  SWAT  data  was  analyzed  with  a  repeated  measures  MANOVA  for  mission  and 
freeze  point.  There  were  no  significant  main  effects  or  significant  interactions  found 
(p>0.05,  Appendix  9,  Table  6).  As  with  the  crew  analysis,  B-1B  experience  was  treated 
as  a  between  subjects  variable  but  was  not  found  to  be  significant  (p>0.05). 


Figure  7.  SWAT  MEAN  -  DSO 


n 


Figure  8.  SWAT  MEAN  BY  MISSION  AND  FREEZE  POINT  -  DSO 


3. 1.5  SWAT  DATA  ANALYSIS  SUMMARY 

To  summarize  the  areas  of  significant  difference.  Table  6  reports  the  SWAT  mean 
difference  and  associated  95%  confidence  interval  for  the  areas  of  statistical  significance. 
The  lower  and  upper  bounds  bracket  the  population  value  consistent  with  this  study’s 
observed  data.  The  95%  confidence  interval  indicates  replications  of  this  study  will 
reproduce  in  the  population  the  same  observed  parameter  within  the  specified  bounds. 

TABLE  6.  SWAT:  MEAN  DIFFERENCE 


p<  .05 


MEAN  DIFFERENCE 
LOWER  LIMIT  95%  C  l. 
UPPER  LIMIT  95%  C.I 


CREW 
MISSION 
SIMPLE  MAIN 
EFFECT  AT 
FREEZE  POINT  2 
6,229* 

I, 295 

II. 162 


oso 

MISSION 
MAIN  EFFECT 


20.858* 

8,261 

33.456 


OSO 

FREEZE  POINT 
MAIN  EFFECT 


23.000* 

7,195 

38.805 
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3.1.6  DUTY 

An  artifact  of  the  study  had  some  WSOs  performing  the  same  duty  (either  OSO  or  DSO) 
for  both  missions,  or  performing  a  different  duty  for  each  mission.  A  separate  analysis 
parsed  out  the  impact  of  duty.  Duty  defined  whether  the  WSO  flew  both  missions  as  a 
dedicated  OSO  or  DSO,  or  flew  one  mission  as  the  OSO  and  the  second  mission  as  the 
DSO  (flexible).  Using  a  Univariate  ANOVA,  SWAT  was  analyzed  across  subjects. 
Subjects  were  identified  as  either  a  dedicated  OSO,  dedicated  DSO  or  performed  both 
functions  (flexible).  The  Univariate  ANOVA  for  the  three  groups  was  significant  (p  < 
0.05,  Table  7).  Also,  the  pair-wise  comparisons  were  significant  (p  <0.05,  Table  8) 
showing  that  workload,  in  general,  is  greatest  for  the  OSO,  as  expected,  and  that  the 
difference  between  dedicated  and  flexible  groups  was  due  to  the  greater  OSO  workload. 
When  combining  the  OSO  and  DSO  dedicated  subjects  into  one  group  and  comparing 
that  group  to  the  flexible  group,  duty  was  not  significantly  different  between  the  two 
groups  (Table  9).  Therefore,  it  is  unlikely  duty  impacted  the  results  noted  in  the  previous 
sections. 


TABLE  7.  UNIVARIATE  ANOVA  -  DUTY  DEPENDENT  MEASURE:  SWAT 

WSO 


Source 

Type  IV  Sum  of  Squares 

df 

Mean  Square 

F 

Sig. 

Corrected  Model 

9839.423 

2 

4919.711 

10.088 

.000 

Intercept 

59628.748 

1 

59628.748 

122.271 

.000 

DUTY 

9839.423 

2 

4919.711 

10.088 

.000* 

Error 

59984.426 

123 

487.678 

Total 

130860.660 

126 

Corrected  Total 

69823.849 

125 

*p<0.01 


TABLE  8.  PAIRWISE  COMPARISONS:  WSO  DUTY 


Dependent  Variable:  SWAT 

Mean  Difference  (l-J) 

(1)  DUTY  (J)  DUTY 

Std.  Error  Sig. 

95%  Confidence  Interval  for 
Difference 

Lower  Bound 

Upper  Bound 

1 

2 

20.717 

4.938  .000 

8.732 

32.703 

3 

3.803 

4.774  1.000 

-7.785 

15.391 

2 

1 

-20.717 

4.938  .000 

-32.703 

-8.732 

3 

-16.914 

4.774  .002 

-28.502 

-5.326 

3 

1 

-3.803 

4.774  1.000 

-15.391 

7.785 

2 

16.914 

4.774  .002 

5.326 

28.502 

Based  on  estimated  marginal  means 

The  mean  difference  is  significant  at  the  .05  level. 


15 


TABLE  9.  UNIVARIATE  ANOVA  -  DUTY  WSO  DEDICATED  VS  WSO 

FLEXIBLE 


Dependent  Variable:  SWAT 


Source 

Type  IV  Sum  of  Squares 

df 

Mean  Square 

F 

Sig. 

Corrected  Model 

1255.126 

1 

1255.126 

2.270 

.134 

Intercept 

61232.278 

1 

61232.278 

110.733 

.000 

NEWGRP 

1255.126 

1 

1255.126 

2.270 

.134 

Error 

68568.722 

124 

552.974 

Total 

130860.660 

126 

Corrected  Total 

69823.849 

125 

3.2  SITUATIONAL  AWARENESS  DATA  ANALYSIS 

Situational  Awareness  was  defined  as  a  binary  event  -  either  they  had  it  or  they  didn't. 
This  binary  data  was  generated  by  comparing  the  answers  given  to  the  Situational 
Awareness  question  during  each  of  the  4  mission  freeze  points,  and  comparing  the 
answer  to  reality  (most  times  with  a  tolerance  band).  If  crewmember's  answer  was  within 
the  reality  band,  the  crewmember  had  Situational  Awareness.  See  Table  10  for  the  SA 
Measures  Tolerance  Bands. 
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TABLE  10.  SA  MEASURES  AND  TOLERANCE  BANDS 
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TABLE  11.  SITUATIONAL  AWARENESS  DATA  TABLES 


Pilot  SA  Data  -  Mission  1  (UTTR) 

Crew  Number 

Freeze  Point 

1  Time  to  TGT 

2  #Hung 
Weapons 

3  #  Weapons 
released 

4  Radar  mode 

1 

No  data 

No  data 

No  data 

2 

No  data 

No  data 

No  data 

No  data 

3 

1 

0 

1 

4 

0 

0 

1 

1 

5 

0 

0 

1 

1 

6 

0 

0 

0 

0 

7 

6 

0 

1 

1 

8 

0 

0 

0 

0 

Summary 

0  of  6  with  SA 

1  of  6  with  SA 

3  of  6  with  SA 

4  of  6  with  SA 

Pilot  SA  Data  -  Mission  2  (Powder  River) 

Crew  Number 

Freeze  Point 

1  Tanker 

Bearing 

2  #  Weapons 
remaining 

3  Ownship 
heading 

4  Time  to 
Waypoint 

1 

No  data 

No  data 

No  data 

No  data 

2 

0 

0 

1 

1 

3 

0 

1 

0 

1 

4 

0 

1 

0 

1 

5 

0 

0 

1 

6 

1 

1 

0 

1 

7 

1 

0 

1 

8 

0 

1 

0 

1 

Summary 

1  of  7  with  SA 

5  of  7  with  SA 

1  of  7  with  SA 

TABLE  11.  continued. 


Crew  Number 

Freeze  Point 

1  Heading  to 
TGT 

2  Altitude 

3  Time  to  go 

4  Active  threat 
status 

1 

No  data 

No  data 

No  data 

No  data 

2 

No  data 

No  data 

No  data 

No  data 

3 

1 

1 

0 

1  ' 

4 

0 

0 

0 

0 

5 

0 

1 

0 

1 

6 

0 

0 

0 

1 

7 

0 

0 

1 

1 

8 

1 

0 

1 

1 

Summary 

2  of  6  with  SA 

2  of  6  with  SA 

2  of  6  with  SA 

5  of  6  with  SA 

Co-Dilot  SA  Data  -  Mission  2  (Powder  River) 

Crew  Number 

Freeze  Point 

1  Tanker 
altitude 

3  Time  to  go 

3  Ownship 
airspeed 

4  Cross-track 

1 

No  data 

No  data 

No  data 

No  data 

2 

0 

0 

1 

0 

3 

1 

1 

0 _ 

4 

1 

1 

0 

5 

1 

0 

1 

0 

6 

1 

1 

0 

0  n 

7 

1 

1 

1 

1 

8 

1 

0 

0 

6  of  7  with  SA 

2  of  7  with  SA 

5  of  7  with  SA 

BiUMiim1 

OSO  SA  Data  -  Mission  1  (UTTR) 

Crew  Number 

Freeze  Point 

1  Time  to  TGT 

2#  Hung 
Weapons 

3  #  Weapons 
released 

1 

No  data 

No  data 

No  data 

2 

No  data 

0 

No  data 

No  data 

3 

0 

0  I 

0 

1 

4 

0 

1 

1 

1 

5 

0 

1 

1 

1 

6 

0 

1 

1 

1  1 

7 

1 

1 

1 

8 

0 

1 

0 

1 

Summary 

1  of  6  with  SA 

5  of  7  with  SA 

3  of  6  with  SA 

6  of  6  with  SA 
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TABLE  11.  continued. 


OSO  SA  Data  -  Mission  2  (Powder  River)  | 

Crew  Number 

Freeze  Point 

1  Tanker 

Bearing 

2  #  Weapons 
remaining 

3  Ownship 
heading 

4  Time  to 
Waypoint 

l" 

No  data 

No  data 

No  data 

No  data 

2 

0 

1 

1 

0 

3 

0 

0 

1 

4 

0 

1 

1  1 

0 

5 

0 

1 

1 

0 

6 

0 

1 

1 

0 

7 

0 

0 

0 

8 

0 

1 

■ 

0 

Summary 

0  of  7  with  SA 

5  of  7  with  SA 

4  of  7  with  SA 

1  of  7  with  S  A 

DSO  SA  Data  -  Mission  1  (UTTR) 

Crew  Number 

Freeze  Point 

1  Threat  type 
and  number 

2  #  Decoy 
Status 

3  #  Threat 
range 

4  Threat  mode 

1 

No  data 

No  data 

No  data 

No  data 

2 

No  data 

No  data 

No  data 

No  data 

3 

1 

1 

1 

1 

4 

1 

1 

"l 

■ 

5 

1 

1 

0 

1 

6 

0 

1 

0 

1 

7 

1 

1 

1 

1 

8 

1 

1 

1 

1 

5  of  6  with  SA 

6  of  6  with  SA 

4  of  6  with  SA 

DSO  SA  Data  -  Mission  2  (Powder  River) 

Crew  Number 

Freeze  Point 

1  Ownship 
altitude 

2  #  Decoy 
status 

3  Threat  type 

4  Number  of 
threats 

1 

No  data 

No  data 

No  data 

No  data 

2 

0 

1 

0 

1 

3 

0 

1 

0 

0 

4 

1 

1 

0 

5 

1 

1 

1 

0 

6 

1 

1 

0 

7 

1 

1 

0 

0 

8 

0 

1 

1 

0 

3  of  7  with  SA 

7  of  7  with  SA 

3  of  7  with  SA 

1  of  7  with  SA 
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3.3  QUESTIONNAIRE  DATA  ANALYSIS 
See  Appendix  2. 

All  rating  scales  were. 


a.  Totally  Acceptable 

b.  Very  Acceptable 

c.  Somewhat  Acceptable 

d.  Borderline 

e.  Somewhat  Unacceptable 

f.  Very  Unacceptable 

g.  Totally  Unacceptable 

An  “a.”  was  the  high  score  of  7,  and  “g.”  was  the  low  score  of  1.  A  question’s  ratings 
were  averaged  across  crewmembers  and  rounded  to  the  nearest  whole  number.  This 
whole  number  determined  the  final  rating.  For  example,  an  average  rating  of  4.07, 
rounded  to  the  whole  number  4,  meant  that  a  “Borderline”  rating  was  reported.  Another 
example  would  be  an  average  rating  of  3.81  rounded  to  the  whole  number  4:  this  would 
also  result  in  a  “Borderline”  rating.  A  rating  frequency  histogram  is  in  Appendix  2  with 
each  question. 

The  results  of  Rank  Ordering  potential  upgrades  are  reflected  in  the  order  they  are 
reported.  That  is,  if  the  1st  item  on  the  list  in  the  questionnaire  is  “Data  Link”,  then  that 
item  received  the  most  votes,  the  2nd  item  “Color  Heads  Down  Display”  came  in  2  ,  etc. 

All  crew  comments  are  reported  in  Appendix  2  verbatim. 
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4.0  DISCUSSION 


4.1  SWAT  DISSCUSSION 

At  the  crew  level,  there  were  no  significant  main  effects  for  mission  and  freeze  point; 
however,  there  was  a  significant  interaction  between  mission  and  freeze  point.  More 
detailed  analyses  revealed  that  there  was  a  significant  difference  in  SWAT  (p<0.05) 
between  Missions  1  and  2  at  freeze  point  2.  At  Mission  1,  freeze  point  2,  there  was  a 
JDAM  hung  store,  and  for  Mission  2,  freeze  point  2  there  was  a  WCMD  bomb  drop.  The 
higher  workload  associated  with  Mission  1,  freeze  point  2  has  implications  for  the  more 
complex  OSO  task  as  noted  below. 

There  were  no  significant  findings  for  the  Flight  Crew  (i.e.,  pilots  &  copilots).  Although 
the  B-1B  ERS  is  a  real  time,  high  fidelity  system,  realistic  combat  mission  conditions 
were  not  feasible  for  the  study.  For  example,  there  was  no  visual  system  so  the  co-pilot 
did  not  scan  the  horizon  with  NVGs,  or  the  accomplishment  of  periodic  checklists  was 
minimal.  Also,  the  simulation  took  place  under  pristine  conditions,  that  is,  there  was  no 
severe  weather  or  aircraft  malfunctions.  Based  on  the  conditions  during  the  test,  the 
controls  and  displays  that  were  available  to  the  flight  crew  appeared  to  provide  an 
acceptable  interface  such  that  workload  was  at  an  acceptable  level.  Pilot  comments  for 
improving  the  pilot-vehicle  interface  are  reported  in  the  questionnaire  section  (Appendix 
2). 

The  major  workload  study  findings  occurred  with  the  OSO.  There  was  a  significant 
difference  in  workload  between  Mission  1  (JDAM)  and  Mission  2  (WCMD)  for  the 
mission  and  freeze  point  MANOVA.  Also,  there  was  significant  and  strong  Eta  Squared 
or  association  between  SWAT  score  differences  in  mission  and  freeze  point  that  will 
generalize  to  the  population.  The  linearity  for  freeze  point  indicates  that  workload 
increased  as  the  mission  progressed.  The  mean  difference  between  missions  indicates 
that  the  workload  for  Mission  1  (JDAM)  was  twice  the  workload  for  Mission  2  (WCMD) 
(40.45  vs.  20.43).  The  SWAT  value  of  40.45  represented  an  average  mission  workload 
value.  Air  Force  Research  Laboratory,  (the  SWAT  developer),  has  determined  that  a 
group  mean  SWAT  score  of  40  represents  a  “red  line”  (i.e.,  it  is  an  indicator  that 
performance  may  start  to  degrade).  Furthermore,  the  means  indicate  that  the  workloads 
associated  with  freeze  points  2,  3,  and  4  for  Mission  1  (JDAM  scenario)  exceed  the 
workload  “red  line”  (46.27,  41.84,  and  51.76  respectively). 

The  WSOs  had  their  choice  to  be  dedicated  to  one  position  or  to  switch  stations.  The 
majority  of  WSOs  (10  versus  6)  had  a  preference  for  one  position  over  the  other.  There 
was  concern  that  the  option  to  change  stations  might  impact  the  crew-level  SWAT  scores. 
The  ANOVA  comparing  the  three  groups  (OSO  dedicated,  DSO  dedicated,  and  Flexible) 
found  duty  did  make  a  significant  difference  in  the  SWAT  scores.  In  further  analysis, 
combining  the  “OSO  dedicated”  and  “DSO  dedicated”  into  one  group  and  comparing  it 
with  the  “Flexible”  group  resulted  in  no  significant  difference  (p>0.05).  This  result 
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alleviates  any  concern  that  duty  impacted  the  crew  SWAT  scores;  however,  future  study 
designs  should  consider  restricting  the  WSO  to  one  station  to  improve  the  power  of  the 
study. 

This  study  did  not  address  system  performance  (i.e.,  bombing  accuracy,  skill  in 
avoidance  of  lethal  threats,  etc).  The  Block  D,  E,  and  F  displays  and  their  interfaces  were 
evaluated,  not  the  system’s  effectiveness.  Furthermore,  combat  conditions  were  not  part 
of  the  simulation  and,  in  particular,  the  flight  deck  did  not  accomplish  their  typical  in¬ 
flight  checklist  procedures,  and  vigilance  activities. 

The  study  data  results  indicate  that  for  the  crew  level,  there  is  no  significant  difference  in 
workload  between  the  JDAM  and  WCMD  missions;  however,  there  is  a  significantly 
higher  workload  associated  with  freeze  point  2.  For  the  JDAM  mission,  freeze  point  2 
was  a  JDAM  hung  store. 

Looking  at  the  workload  of  each  crew  position,  the  study  data  suggests  that  the  flight 
crew  (pilot  and  co-pilot)  and  the  DSO  workload  levels  are  acceptable.  The  OSO  has  the 
highest  workload  and  it  may  exceed  an  acceptable  workload  capacity.  This  conclusion  is 
in  agreement  with  the  questionnaire  responses.  Although  the  pilots  made 
recommendations  for  improving  the  CDU  and  recommended  other  necessary  information 
to  improve  their  flight  deck  situation  awareness,  in  general,  the  results  found  that  the 
Pilot  Vehicle  Interface  (PVT)  was  acceptable.  The  same  is  true  for  the  DSO.  For  the 
most  part,  the  DSO’s  controls  and  displays  were  acceptable.  Unlike  their  crewmates,  the 
OSOs  rated  their  displays  (JDAM,  WCMD,  and  LAR  displays)  marginal  to  somewhat 
acceptable,  as  expected,  given  the  complexity  of  the  OSO  tasks. 

Based  on  the  workload  results  of  this  study,  the  OSO’s  workload  is  excessive  for  the 
JDAM  scenario.  Starting  with  freeze  point  2  (JDAM  hung  store)  until  the  end  of  the 
mission,  the  OSO’s  workload  as  indicated  by  the  SWAT  scores,  exceeds  recommended 
limits.  Control  and  display  improvements  to  the  Block  E  JDAM  and  LAR  displays  are 
recommended  to  reduce  the  OSO’s  workload.  The  biggest  return  on  invested  acquisition 
dollars  will  come  from  upgrades  for  the  OSO.  It  should  be  noted,  however,  that  changes 
to  the  OSO  station  may  impact  the  DSO  as  well. 


4.2  SITUATIONAL  AWARENESS  (SA)  DISCUSSION 

4.2.1  PILOT  &  COPILOT  SITUATIONAL  AWARENESS  DISCUSSION 

Collapsing  across  Pilots,  Copilots,  Freeze  Points,  and  Missions,  the  subjects  had 
Situational  Awareness  only  45%  of  the  time.  This  may  be  attributed  to  the  fact  that  this 
was  a  simulation  and  the  pilots  and  copilots  sometimes  seemed  bored  or  apathetic  (e.g., 
two  crews  “crashed”  the  aircraft  during  a  mission).  Further,  they  might  have  been  under¬ 
challenged  by  the  simulator  (e.g.,  only  CDU  was  “new”,  no  motion  base,  visual  system, 
use  of  check  lists,  in-flight  emergencies,  etc.).  And  lastly,  the  SA  Questions  may  have 
been  inappropriate  or  meaningless  for  the  activity  associated  with  that  freeze  point. 
Otherwise  the  SA  scores  in  Table  10  speak  for  themselves. 
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Inappropriate  SA  questions  are  perhaps  a  major  flaw  in  the  SAGAT  technique  in  the  full 
mission  simulation  arena.  Questions  asked  at  the  freeze  point  are  supposed  to  be 
randomly  chosen  from  a  very  long  list  of  possibilities.  Experimentally,  this  is  good 
technique,  but  a  randomly  chosen  question  may  be  totally  meaningless  in  the  context  of 
that  particular  mission/ffeeze  point.  Therefore,  if  the  crew  member  lacks  SA,  is  this  due 
to  a  system  design  flaw,  such  as  a  lack  of  appropriate  SA  information,  or  to  a  question 
that  is  meaningless  when  taken  in  the  mission/freeze  point  context?  In  other  words,  there 
is  no  point  in  knowing  that  particular  SA  information  at  that  point  in  time.  The  study 
attempted  to  reconcile  this  conflict  by  trying  to  give  contextually  meaningful  SA 
questions,  but  still  trying  to  quasi-randomly  select  the  SA  question.  It  is  believed  that 
this  was  only  partially  successful  and  may  account  for  some  unknown  portion  of  the 
“lack  of  SA”.  Similar  comments  may  apply  to  the  OSO  and  DSO  S  A  scores. 

No  meaningful  Situational  Awareness  data  could  be  gotten  from  either  the  first  crew’s 
and  sometimes  the  second  crew’s.  Pilots  and  Copilots.  This  was  because  of: 

a.  B-1B  ERS  software  problems,  such  as  a  proper  (software)  triggering  of  freeze 
points, 

b.  A  lack  of  instructional  emphasis  on  strictly  meeting  the  Estimated  Time  of  Arrival 
(ETA)  for  the  waypoints.  The  freeze  points  were  automatically  time  triggered  with 
a  clock  start  time  based  on  the  unfreezing  of  the  simulator  (i.e.,  simulator 
“takeoff’),  and 

c.  Improper  formulation  of  Situational  Awareness  questions  that  were  meaningful  in 
the  context  of  the  current  mission  freeze. 

4. 2. 2  OSO  SITUA  TIONAL  A  WARENESS  DISCUSSION 
Collapsing  across  Freeze  Points  and  Missions,  48%  of  OSOs  had  S  A. 

No  meaningful  Situational  Awareness  data  could  be  obtained  from  the  first  two  crew’s 
OSOs  for  the  same  reasons  as  discussed  in  paragraph  4.2. 1  PILOT  &  COPILOT 
SITUATIONAL  AWARENESS  DISCUSSION. 

4. 2. 3  DSO  SITUA  TIONAL  A  WARENESS  DISCUSSION 
Collapsing  across  Freeze  Points  and  Missions,  65%  of  DSOs  had  SA. 

No  meaningful  Situational  Awareness  data  could  be  obtained  from  the  first  crew’s  DSOs 
for  the  same  reasons  as  discussed  in  paragraph  4.2.1  PILOT  &  COPILOT 
SITUATIONAL  AWARENESS  DISCUSSION. 


24 


4.3  QUESTIONNAIRE  DISCUISSION 

4.3.1  PILOT  AND  COPILOT  RESPONSES 

According  to  the  questionnaire,  14  out  of  16  pilots  and  copilots  thought  some  changes 
need  to  be  made  to  the  CMNS  CDU.  The  acceptability  of  the  CDU  Flight  Plan  interface 
was  rated  as  borderline.  Internal  crew  and  external  world  coordination,  and  flight 
situational  awareness  were  rated  as  somewhat  unacceptable.  Targeting  and  tanker 
rendezvous  information  was  rated  borderline.  Out  of  12  choices,  the  top-three  rated 
items  were  Data  Link,  HUD,  and  Color  Heads  Down  Displays  in  terms  of  operational 
usefulness  for  future  Block  upgrades. 

4.3.2  OSO  RESPONSES 

The  WCMD  and  JDAM  display  formats  were  rated  as  borderline.  DLST  was  rated  as 
very  acceptable.  LAR  displays  and  targeting  information  were  rated  as  somewhat 
unacceptable.  Eight  out  of  eleven  OSOs  were  able  to  use  the  LAR  display  to  steer  to  the 
upcoming  weapons  LAR(s).  Only  four  out  of  ten  OSOs  were  able  to  follow  the  sequence 
of  weapons  releases  using  the  LAR  display.  Inter-Crew  Coordination  and 
Communication,  and  Coordination  with  the  outside  world  was  rated  as  somewhat 
acceptable.  Target  Situational  Awareness  was  rated  borderline.  The  top-three  out  of 
seven  most  operationally  useful  improvements  would  be  Data  Link,  Color  Heads  Down 
Display,  and  Moving  map  (i.e.,  the  same  as  the  DSO). 

4.3.3  DSO  RESPONSES 

The  ASQ-185  RWR  Controls  and  Displays  were  rated  somewhat  acceptable.  The  FOTD 
CDU,  the  ALR-56M,  inter-crew  coordination,  EW  MFD  formats,  and  Threat  Situational 
Awareness  were  rated  very  acceptable.  Communications,  the  CDU  and  ASQ-184 
Combination,  and  RFS/ECM  panel  were  rated  somewhat  acceptable.  The  top-three  out 
of  seven  operationally  useful  future  upgrades  were  Data  Link,  Color  Heads  Down 
Displays,  and  Moving  Map  (i.e.,  the  same  as  the  OSO). 


4.4  GENERAL  LAUNCH  ACCEPTABILITY  REGION  (LAR)  DISCUSSION 

a.  From  SWAT  we  can  see: 

Based  on  the  workload  results  of  this  study,  the  OSO’s  workload  is  excessive  for 
the  JDAM  scenario.  This  excessive  workload  is  significantly  compounded  by  a 
hung  store.  Starting  with  freeze  point  2  (JDAM  hung  store)  until  the  end  of  the 
mission,  the  OSO’s  workload,  as  indicated  by  the  SWAT  scores,  exceeds 
recommended  limits. 


b.  From  the  questionnaire  we  can  see: 

The  WCMD  and  JDAM  display  formats  (e  g.,  D  &  E  Pages)  were  rated  as 
borderline.  LAR  displays  (EB  Page)  and  targeting  information  were  rated  as 
somewhat  unacceptable.  Eight  out  of  eleven  OSOs  were  able  to  use  the  LAR 
display  to  steer  to  the  upcoming  weapons  LAR(s).  Only  four  out  of  ten  OSOs 
were  able  to  follow  the  sequence  of  weapons  releases  using  the  LAR  display. 
Target  Situational  awareness  was  rated  borderline. 

c.  From  Observation  and  the  Video  Tape  we  can  see  (see  TABLES  12.  A.  & 
12.B.): 
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TABLE  12. A.  UTTR  -  LAR  OBSERVATIONS 
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Off  Line  On  Line 
76.70%  89.30% 
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TABLE  12.B.  POWDER  RIVER  -  LAR  OBSERVATIONS 
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Range  = 

0  to  192 


1)  The  LAR  was  used  an  average  of  33  seconds  per  bomb  run  for  the  three 
JDAM  and  the  three  WCMD  bomb  runs.  The  LAR  usage  range  was  zero  to 
192  seconds  per  bomb  run.  For  12  of  48  or  25%  of  the  bomb  runs,  the  LAR 
was  not  used  by  the  OSO  at  all. 

2)  The  OSO  used  the  LAR  only  briefly  before  turning  to  other  more  important 
pages  such  as  the  Weapons  Summary  (D  Page)  or  to  the  Target  Summary 
page  (E  page). 

3)  All  six  bomb  runs  were  at  24,000  feet  MSL.  At  that  altitude,  no  matter  how 
far  the  crew  got  off  of  the  planned  route  or  “black  line”,  it  seemed  that, 

a.  as  long  as  they  were  generally  heading  for  the  target  steerpoint, 

b.  and  the  target  was  a  horizontal  target  (in  this  study  there  were  no  vertical 
targets), 

c.  and  the  crew  hadn’t  already  flown  completely  through  or  beyond  the 
LAR, 

Then  LAR  was  always  achieved. 

4)  It  seemed  to  greatly  help  the  OSO  if  the  DSO  stepped  in  to  help  the  flight 
deck  crew  navigate  while  the  OSO  dealt  with  the  hung  store.  Further,  it 
seemed  to  help  considerably  if  the  OSO  looked  at  the  DSO’s  “stick  map”  on 
the  TSD  to  keep  general  situational  awareness  of  the  aircraft’s  navigational 
situation. 

5)  Overall,  the  LAR  display  was  beneficial  when  used  properly  in  this 
simulation.  During  testing,  the  LAR  display  (EB  page)  was  used  between  0 
and  192  seconds  per  bomb  run  with  an  average  of  33  seconds  per  bomb  run. 
During  these  runs,  it  was  noted  that  under  use  as  well  as  over  use  tended  to 
negatively  impact  bombing  performance.  It  was  felt  the  crew’s  best 
performances  were  shown  when  the  majority  of  information  was  taken  from 
the  Target  Summary  (E  page)  page  while  incorporating  the  LAR  information 
for  fine-tuning  their  bomb  run.  It  was  noticed,  however,  that  there  were  times 
when  OSO  information  was  limited  and  resulted  in  rapid  decline  in  OSO 
situational  awareness  (SA).  This  situation  was  mostly  prevalent  during  multi¬ 
tasking  such  as  navigation  to  bomb  run  combined  with  trying  to  resolve  a 
hung  store.  In  these  cases,  not  only  was  the  decline  in  SA  rapid,  but  the 
process  of  regaining  SA  was  extremely  slow.  On  one  occasion,  the  OSO  was 
so  disoriented  that  the  crew  had  navigated  past  the  3rd  bomb  run  before  the 
OSO  even  realized  that  they  had  not  dropped  any  weapons  on  the  second 
bomb  run.  Next  are  some  of  the  factors  that  contribute  to  the  OSO’s  lack  of 
SA. 

6)  Near-Mode-Ranging  -  While  navigating  to  the  LAR  while  in  far-mode- 
ranging,  the  Target  Summary  (E  page)  page  shows  an  estimated  time  to 
achieve  LAR.  However,  when  switching  to  near-mode-ranging,  the  target 
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summary  gives  actual  time  to  achieve  LAR.  In  order  to  get  this  actual  time 
the  aircraft  must  be  headed  towards  the  LAR.  If  the  aircraft  is  flown  off  the 
“black  line”,  then  the  time  to  go  (TTG)  to  LAR  reverts  to  a  default  value  of 
59:59.  This  signifies  the  max  value  of  TTG  of  59  minutes  and  59  seconds. 

As  a  result,  the  OSO  is  left  without  half  of  the  needed  bombing  information 
(TTG  and  Heading)  during  a  critical  phase  of  flight.  On  several  occasions, 
while  inbound  to  the  target,  the  Pilot  asked  the  OSO  what  the  TTG  to  LAR 
was  and  if  they  were  off  the  “black  line”.  Invariably  the  OSO  s  answer  was  I 
don’t  know”. 

7)  Another  function  of  near-mode-ranging  raises  some  concern.  When  the 
aircraft  enters  the  LAR,  the  TTG  changes  from  time  to  achieve  LAR  to  time 
to  exit  LAR.  The  only  information  showing  this  shift  is  a  RANGE/ZONE 
indication.  Although  the  information  is  available,  it  is  not  immediately 
obvious.  On  a  few  occasions,  when  the  crew  was  turning  back  towards  the 
“black  line”,  the  turn  placed  them  into  the  LAR  and  the  TTG  indication 
changed  from  59:59  to  the  actual  time  to  exit  the  LAR.  The  OSO,  however, 
interpreted  this  as  time  to  achieve  LAR  and  was  surprised  when  the  bay  doors 
started  to  open  and  bombs  dropped. 
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5.0  RECOMMENDATIONS 


5.1  LAUNCH  ACCEPTABILITY  REGION  (LAR)  RECOMMENDATIONS 

Control  and  display  improvements  to  the  Block  E  JDAM  and  LAR  displays  are 
recommended  to  reduce  the  OSO’s  workload.  The  biggest  human  factors  return  on 
investment  will  come  from  upgrades  for  the  OSO. 

a.  An  improved  LAR  display  (EB  Page)  is  needed,  which  gives  more  information 
about  the  LAR.  The  “LAR  bars”  only  give  general  relative  position,  yet  yield  no 
Time  To  Go  (TTG)  or  Distance  information.  As  a  result,  the  usefulness  of  the  EB 
page  for  navigational/horizontal  situational  awareness  is  very  limited. 

b.  Improved  targeting  information  is  needed.  The  target  summary  page  (E  Page) 
needs  distance  and  time  information  at  all  times.  When  the  aircraft  is  steered  off 
of  the  “black  line”  in  near  mode  ranging,  the  OSO  looses  all  distance  and  time 
information  to  go  to  LAR,  leaving  only  a  heading  value.  This  heading  value  only 
gives  guidance  to  the  Target  steerpoint  and  not  to  the  “mouth”  of  a  LAR.  This 
lack  of  information  results  in  a  significant  loss  of  navigational/horizontal 
situational  awareness  at  a  critical  time  for  the  OSO. 

c.  On  the  E  Page,  when  near  mode  ranging  changes  from  far  mode,  if  aircraft  is  not 
pointed  towards  LAR  (i.e.,  off  the  “black  line”),  TTG  changes  from  Actual  Time 
to  the  default  value  of  59:59  minutes.  Give  Estimated  Time  to  LAR,  not  default, 
when  going  from  far  to  near  mode  ranging.  Display  Estimated  TTG  in  reverse 
highlight  (or  in  some  other  distinct  way)  to  distinguish  it  from  Actual  TTG. 


5.2  BASELINE  STUDY  RECOMMENDATIONS 

a.  Improved  hung  store  training  is  needed  for  OSOs. 

b.  A  Digital  Bull,  Bra,  and  Compass  Rose  should  be  added  as  soon  as  possible  to 
theDSO’s  TSD. 

c.  WSOs  have  reservations  about  not  being  able  to  re-identify  a  threat  if  the 
defensive  system  is  wrong.  DSO  threat  re-identification  capability  should  be 
added  to  the  defensive  system’s  controls  and  displays. 

d.  An  ALR-56M  Repeater  should  be  added  to  the  front  flight  station. 

e.  There  seems  to  be  a  tendency  for  WSOs  to  treat  JDAMs  and  WCMDs  as  “stand¬ 
off”  weapons  and  not  as  enhanced  accuracy  weapons.  This  may  stretch  the 
capabilities  of  the  weapons  and  associated  controls  and  displays  to  or  beyond  the 
limit.  Thus,  JDAM  and  WCMD  weapon  capability  training  should  be  given  or 
improved. 
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f.  A  Data  Link,  HUD,  and  Color  Heads  Down  Displays  are  the  top-three  rated 
items  that  should  be  included  in  future  Block  Upgrades  for  the  pilot  and  copilot. 

g.  A  Data  Link,  Color  Heads  Down  Displays,  and  Moving  Map  are  the  top  three 
rated  items  that  should  be  included  in  future  Block  Upgrades  for  the  WSOs. 

h.  During  critical  phases  of  a  mission,  OSO  task  saturation  may  be  high.  This  is 
compounded  by  the  fact  that  the  OSO  is  responsible  for  navigation  as  a  primary 
duty.  A  review  of  current  WSO  procedures  should  be  made  with  an  eye  towards 
the  DSO  becoming  more  involved  in  the  navigation  tasks.  This  may  alleviate  a 
significant  portion  of  the  OSO’s  overall  workload. 
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APPENDICIES 


APPENDIX  1  -  CREW  SCHEDULE,  BASE,  AND  DATA  MISSION  ORDER 


DATE: 

BASE: 

MISSION  ORDER: 

9-10  June 

Tinker  AFB 

1,2* 

10-11  June 

Dyess  AFB 

2,1 

15-16  June 

Robbins  AFB 

1,2 

17-18  June 

Dyess  AFB 

2,1 

22-23  June 

McConnell  AFB 

2,1 

29-30  June 

Robbins  AFB 

1,2 

8-9  July 

Dyess  AFB 

1,2 

12-13  July 

Ellsworth  AFB 

2,1 

*  Mission  1  =  UTTR:  Mission  2  =  Powder  River 


APPENDIX  2  -  QUESTIONNAIRES 


B-1B  Baseline  Study 
Questionnaire 


Crewman  # 


Date:  _ 

(Your  Name,  Rank,  Unit/Base,  Phone  Number  will  be  kept  private  and  will  not  appear  in 
any  reports  on  this  study.) 

NAME: _  RANK:  AGE 


UNIT/BASE  Address: 


DSN  NUMBER: _  E-Mail  Address: 


1.  A.  Pilot/Copilot  Experience 

Current  Qualified  Aircraft:  B-1B  Number  of  Flying  Hrs:  Ave.:  1212.56 

Range:  1 50-2600  hrs 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _ Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _  Number  of  Flying  Hrs _ 

Total  Flying  Hrs:  Ave.  2122.56  Range:371-3650  hrs: 

This  section  reflects  only  PILOT  responses. 

2.  B-1B  Simulator  Time:  Yes:  16  No:  _0 

Type  Sim:  WST.  CPT.  Full  Motion  ERS.  B-1B  SBOOlBffl 
Number  of  Hrs:  Ave.  218,44  Range:  50-700 
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Purpose:  (e.g.  Upgrade  training  for  Block  X):  initial  qual-copilot;  upgrade- 
pilot/instructor  FTU  instructor:  Initial/recurrent/upgrade  training;  Training  and  as 
instructor:  Mission  qualification:  Training  and  unit  currency;  Standard  schoolhouse 
training  and  unit  sims. 


3.  Have  you  had  previous  B-1B  Block  D,  E  and/or  F  experience?  Yes:  4  No:  12 

If  yes,  what  aircraft/simulator(s)?  B-1B  Block  D  Number  ofHrs _ 

4.  Have  you  had  any  AFMSS  Training?  Yes:  ]_  No:  9 

5.  Have  you  had  any  training  on  GMTI?  Yes:  J_  No:  15 

6.  Have  you  had  any  training  on  JDAM?  Yes:  6_  No:  9_  No  response:  1 

7.  Are  you  familiar  with  Launch  Acceptability  Regions  (LARs)?  Yes:  10.  No:  6 

8.  a.  Have  you  ever  participated  in  any  B-1B  studies  or  flight  tests?  Yes:  _7_  No:  9 

b.  If  yes,  describe: 

•WST  software  test; 

•  10FLTS  flight-test  missions; 

•PBAR  for  BDU-33; 

•JSEAD/Live-EX  -  Green  Flag  97-3; 

•Weapons  delivery; 

•Block  D  initial  controls  and  display  (human  factors); 

•Block  D  TD&E 

•Assigned  to  B-l  operational  test  unit,  tested  Blocks  B,  C  and  D 
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I.  Pilot  &  Copilot  Section 


1 .  a.  Does  the  Control  Display  Unit  (CDU)  need  any  changes?  Yesl4  No! 

1  no  response. 

b.  If  yes,  describe: _ 

•  Nav  function  for  TAC  emulation  does  not  operate.  Defeats  briefed  purpose  of  divert  scenario. 
In  Fplan  cannot  scroll  to  previous  pts.  Once  Dir. is  selected,  must  manually  select  Fplan  to 
get  steering 

•  Not  totally  representative  of  Block  D  modified  acft.  For  this  test  -  comm  package  OK.  Nav  - 
doesn’t  work  -  not  TAC  emulation  -  if  try  to  input  ident.  -  kicks  to  window  mode.  STR  page 
works  OK.  Can  input  coordinate  change  to  flight  plan  page  +  (and?)  go  DIR  TO.  index  for 
POWER  PG  CK.  MASTER  PWR  ON  ok.  Everything  else  does  not  need  to  work  for  this  test. 

•  See  #2 

•  All  CDUs  should  be  connected  by  a  data  bus!!!  Should  have  "what  if'  sequence  capability 
(i.e.  what  if  I  go  direct  to  this  point,  how  will  this  affect  my  timing,  track,  target  without 
effecting  the  ins.  track. 

•  Move  Eng.  Start  switches  toward  firewall  away  from  CDU,  or  put  CDU  next  to  firewall.  Needs 
to  be  integrated  into  aircraft 

•  Cant  tell  at  this  point 

•  Have  one  CDU  for  each  pilot 

•  Need  hour/min/sec  on  ETA,  now  just  hr/min  ETA  -  60=00  now  -  need  00  ex.  1900=1860 

•  Fix  the  parallax  error.  Change  pages  so  that  it  displays  the  D#  you're  going  to  on  the  steer 
page 

•  XTRK  L  3.0nm  should  mean  that  you  3nm  left  of  track,  instead  this  system  says  that  you  are 
3nm  right  of  centerline,  very  confusing.  ETA  should  include  seconds  to  go. 

•  Need  seconds  on  ETO  on  STR  page 

•  Color  would  be  nice 

•  I  feel  that  the  ETA  display  should  include  seconds.  When  you  are  trucking  along  at  8  to  9 
miles  a  minute,  it  would  be  great  to  just  look  down  and  see  if  you  are  early  or  late  (to  the 
second)  instead  of  doing  the  needed  math  (carry  over  whole  minutes).  Pilots  want  to  know  if 
what  they  are  doing  with  get  them  to  the  target/destination  on  time 

•  T o/From  switch  for  digital  bull 

•  To  make  the  VSD  heading  marker  work  I  had  to  sequence  to  the  steer  point  on  the  CDU 

•  I  need  to  be  able  to  re-name  the  steer  points 

•  Needs  to  be  able  to  display  Bullseye  Bearing/Distance  in  addition  to  the  current  steerpoint 
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2.  Rate  the  acceptability  of  the  CDU  Flight  Plan  Interface  (Circle  one)  • 

Average  Rating:  4.07  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  4 

c.  Somewhat  Acceptable  3 

d.  Borderline  0 

e.  Somewhat  Unacceptable  6 

f.  Very  Unacceptable  2 

g.  Totally  Unacceptable  0 

1  no  response 

Comments: _ 

•  See  Above 


•  Can  input  flight  plan  change  ok 

•  Add  TTG  or  ETA  to  each  point  or  flight  plan  page 

•  The  pilots  need  to  monitor  closely  the  CDU  flight  plan.  It  does  not  always  sequence 
automatically 

•  See  above;  and  more  CDU  FWD  away  from  the  throttle  quadrant. 

•  Integrate 

•  N/A  -  this  was  not  about  the  CDU  -did  not  get  a  chance  to  evaluate  the  entire  CDU 
capability/T.F.  interface 

•  Add  current  waypoint  #  to  FPI  page 

•  not  attached  to  avionics 

•  Did  not  integrate  with  flight  plan  enough  to  rate 

•  Some  integration  with  WSO  would  be  nice,  however  not  mandatory 

•  Would  be  good  to  interface  with  NAVs  head(?).  Suggest  the  option  of  having  the  OSO 

update  the  pilot's  CDU _ _ _ _ _ 
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3.  Rate  the  acceptability  of  the  Inter-Crew  Coordination  ( Circle  one)  • 

Average  Rating:  3.81  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  4 

c.  Somewhat  Acceptable  2 

d.  Borderline  2 

e.  Somewhat  Unacceptable  3 

f.  Very  Unacceptable  5 

g.  Totally  Unacceptable  0 


Comments: _ 

•  Required  much  cockpit  chatter  to  fill  in  SA  holes 

•  More  experienced  crew  allows  better  SA  -  information  flow  was  good  to  help  diagnose  sim 
"isms",  help  trouble  shoot  problems. 

•  LAR  presentation  to  pilots  should  be  added  to  minimize  comms. 

•  Pilots  need  better  idea  where  LAR  is  and  how  maneuvers  affect  LAR 

•  The  front  station  has  actually  lost  SA  -  no  indication  when  weapons  release  is  about  to/is 
happening 

•  Presentation:  No  one  has  a  good  understanding  of  when  the  weapons  are  releasing,  how 
many  have  released?  When  were  cleared  to  maneuver. 

•  need  to  integrate  CNMS  with  ACU 

•  way  too  much  talking,  could  interfere  with  radio  communications 

•  CPT  has  limitations  such  as  WSO  displays  for  bank  and  altitude  inoperative  -  makes  for 
confusion. 

•  The  aft  station  needs  to  be  able  to  work  the  CNMS  CDU  from  the  aft  station  and  also  see 
what  the  pilots  are  doing,  so  we  are  in  the  same  sheet  of  music. 

•  The  limited  SA  requires  additional  (from  present  day)  crew  coordination.  More  chatter. 

•  Need  to  have  common  displays  which  will  allow  less  interphone  chatter _ 
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4.  What  changes,  if  any,  might  be  made  to  improve  crew  coordination? 

Comments:  _ _ _ _ _ 

•  More  information  available  and  a  better  crew  pre-brief  so  as  to  know  what  to  expect  during 
bomb  runs.  Understanding  this  is  a  limitation  of  the  sim  profile 

•  Make  sure  crew  talks  briefly  about  mission  prior  to  going  into  box.  Ensure  flight  plan  reviewed 
-  what  configuration  are  you  going  fly,  crunch  points,  options,  potential  problems,  etc. 

•  Threat  display  &  Bull  information  displayed  in  front  station 

•  Digital  Bull  information  on  the  pilot's  VSD  would  be  outstanding  improvement:  just  2  numbers 
(bearing  and  range  of  the  B-1 B  to  the  Bull)  would  be  extremely  helpful  to  pilots  and  would  cut 
down  on  inter-cockpit  communication. 

•  RWR,  LAR  presentation  to  pilots 

•  Integrate  CDU  -  Put  moving  LAR  superimposed  over  TA  display  with  appropriate  ranges 

•  Block  D  has  no  LAR  information  for  front  or  back  i.e.  unless  we  turn  at  the  target,  we  may  not 
know  if  we  are  in/near  the  LAR  i.e.  off-axis  release  planning.  Block  E/F  still  does  not  give  the 
pilot  station  any  clue  to  when  a  weapon  is  about  to  release.  No  way  to  back-up  the  WSOs. 

•  Modify/replace  SMS  panel  (with  MFD)  to  display  number  of  weapons  being  released.  TTG  to 
1st  release,  indicate  last  release.  The  MFD  should  also  have  a  RWR  function,  weather  radar 
page. 

•  RWR  up  front.  Moving  map  up  front 

•  Add  moving  map  display 

•  Direction/timing/and  distance  to  LAR  on  bottom  center  of  VSD  would  aid  pilot  SA,  and 
eliminate  unnecessary  comms  during  critical  phases  of  flight  as  well  as  adding  a  RAW 
repeater  scope.  Along  with  a  separate  Bullseye  repeater  for  pilot  and  copilot,  capable  of  low- 
color  weather  radar 

•  Training  on  system  at  each  station.  Nice  to  know  what  information  available,  LAR  information 

•  Allow  NAV  to  update  pilot's  CDU 

•  Moving  map  with  data  link  up  front,  so  the  pilots  can  see  the  threat,  LAR,  course  line,  etc. 

•  Improve  SA  for  each  station 

•  In  the  sim  the  VSD  must  be  accurate 

•  OSO  needs  an  ADI 

•  A  phone  call  tone  is  unacceptable  as  a  threat  warning  tone. _ _ 


5.  Rate  the  acceptability  of  the  Coordination  with  the  Outside  World  (e.g.,  radios) 

( Circle  one)  l 

Average  Rating:  3.38  (Highest  score  -  7;  lowest  1) 


,  a.  Totally  Acceptable  0 

b.  Very  Acceptable  0 

c.  Somewhat  Acceptable  4 

d.  Borderline  1 

e.  Somewhat  Unacceptable  4 

f.  Very  Unacceptable  4 

g.  Totally  Unacceptable  0 

3  no  response 

Comments: 


•  Due  to  limitations  of  sim,  there  was  literally  no  outside  comm.  If  we  were  part  of  a  large 
package,  as  pre-briefed,  there  would  be  constant  radio  chatter  that  would  have  to  be  sorted 
through  and  reacted/responded  to.  This  would  be  an  SA  builder  and  detractor  depending  on 
crew  coordination  required  at  that  time 

•  ILST  -  improved  SA  is  good.  Continued  troubleshooting  of  current  Block  D  radio  problems. 
Data  link  would  be  nice.  Real  time  info  is  good. 

•  Need  Accurate,  quick  ability  to  see  threats  and  mission  information  (Link-16  type 
information).  Also  in  dire  need  of  a  digital  bull. 

•  Needs  data  link 

•  We  really  did  not  do  any 

•  Unknown 

•  Not  much  work  here  in  the  sim 

•  Block  D  radios  are  unacceptable  in  trans/recp.  Also  must  have  digital  bull  at  every  position. 
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6.  Rate  the  acceptability  of  the  Flight  Situational  Awareness  (Circle  one) ! 

Average  Rating:  3.56  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  0 

c.  Somewhat  Acceptable  4 

d.  Borderline  1 

e.  Somewhat  Unacceptable  5 

f.  Very  Unacceptable  3 

g.  Totally  Unacceptable  1 

2  no  response 

Comments:  _ _ _ 


•  Limitations  of  sim  profile  and  see  above 

•  Info  flow  was  pretty  good  between  stations.  ILST  info  was  good  for  tanker  rendezvous. 

•  See  #  5.  Also,  front  seat  workload  would  be  drastically  reduced  by  adding  MFDs  and  a 
HUD.  The  B-1  has  very  poor  visibility  and  SA  is  made  worse  by  having  to  keep  head 
down. 

•  A  RWR  scope  for  pilots  would  be  very  helpful  for  situational  awareness.  It  could  currently 
go  where  the  Aux,  Master  Caution  Panel  is  and  the  Aux  panel  could  go  on  the  left  side  of 
the  pilot's  HSI  so  the  co-pilot  can  see  it  also. 

•  Need  RWR,  SMS  infonmation  to  reduce  crew  communication 

•  Need  moving  map  /  data  link  /  RWR  /  weapons  data 

•  No  moving  map  display  or  threat  picture 

•  See  comments  for  #4 

•  CDU  increases  SA  100%! 

•  I  think  CPT  limitations  were  the  problem 

•  Cross  track  is  good,  but  a  picture  is  worth  1000  words.  It  is  nice  to  see  if  your  ADF 
maneuver  is  going  to  work. 

•  As  a  pilot  I  am  now  almost  totally  out  of  the  bombing  SA  loop.  Any  time  you  force  crew 

coordination  you  limit  the  crew's  ability  to  perform. _ _ _ 


7.  Rate  the  acceptability  of  the  Targeting  Information  controls  &  displays  (e.g.,  Launch 
Acceptability  Region  (LAR)  (Circle): 

Average  Rating:  4.47  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  3 

b.  Very  Acceptable  3 

c.  Somewhat  Acceptable  1 

d.  Borderline  4 

e.  Somewhat  Unacceptable  1 

f.  Very  Unacceptable  1 

g.  Totally  Unacceptable  2 

1  no  response 

Comments: 


•  Requires  constant  crew  coordination  OSO  must  give  pilots  a  PAR  to  LAR.  Pilots  have  No 
information  as  to  heading  and  distance  to  LAR. 

•  No  SA  for  pilot  other  than  what  he  hears  from  OSO.  In  a  cluttered  radio  environment  (Red 
Flag)  -  unacceptable.  Eventually  need  visual  indication  up  front  as  to  where  you  are  in 
relationship  to  LAR  and  how  to  quickly  to  correct. 

•  Very  difficult  to  visualize  where  and  how  to  get  to  LAR 

•  Pilots  have  no  SA  on  the  LAR 

•  From  pilot's  perspective 

•  Need  LAR  where  SRAM  Safe  and  In  Range  (SAIR)  was  on  bomb  panel.  ERS  bomb  nav 
panel  for  Bomb  mode  and  VSD  and  doors  need  to  work  like  airplane.  Don't  need  master 
caution  light  upon  bay  selection 

•  Cannot  tell  if  OSO  is  in  bomb  mode,  if  WIUs  are  powered.  Block,  pilots  get  a  "Bomb"  mode 
in  USD  when  a  target  is  selected  and  bomb  mode  is  on.  SMS  panel  give  no  SA  except 
doors  are  open  99  goes  out.  In  the  heat  of  battle  the  pilots  cannot  help  out  the  WSOs. 

•  No  display  /  information  up  front. 

•  No  SA  at  all  -  need  LAR  displayed  on  moving  map  and  continually  updated 

•  Need  more  LAR  information,  range,  TTG 

•  see  comments  for  #4 

•  If  your  IP  to  TGT  is  0 - 1 

but  you  actually  get  the  weapon  off  here  there  is  no  reason  to  go  to  target.  So  this  mission 
planning  needs  to  be  reconsidered 

•  There  truly  was  none  up-front;  however  there  needs  to  be. 

•  Need  LAR  steering  at  front  station _ 
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8.  Rate  the  acceptability  of  the  Threat  Information  (e.g.,  Threat  Bearing)  (Circle  one): 

Average  Rating:  3.87  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  3 

b.  Very  Acceptable  2 

c.  Somewhat  Acceptable  1 

d.  Borderline  1 

e.  Somewhat  Unacceptable  3 

f.  Very  Unacceptable  2 

g.  Totally  Unacceptable  2 

1  no  response 

Comments: _ _ _ 

•  Need  audio 


•  Good  DSO  is  important.  Threat  SA  seemed  to  be  pretty  good  for  this  test. 

•  See  #  5 

•  See  #6 

•  The  addition  of  RWR  to  front  station  will  enhance  SA 

•  Still  no  SA  /  New  guy  tone  -  outstanding 

•  No  display  up  front. 

•  Needs  RWR  up  front.  Need  expendable  control  up  front 

•  No  information  up  front  except  interphone  -  too  much  likelihood  of  stepping  on  each  other 

•  See  comments  for  #4 

•  With  ALR-56M  repeater 

•  DSO  did  the  work.  He  was  good: 

•  Move  threat  repeater  down  below  the  Aux  Caution  panel  on  the  pilot's  side. 

•  Threat  audio  tone  gives  no  capability  to  ID  a  threat  with  a  visual  RWR  indication.  Air-to- 
Air,  Air-to-Ground,  AQ,  TT,  IL? 

(Additionally,  respondent  indicated  this  as  "e.  Somewhat  Acceptable"  for  simulator  range 
and  bearing  and  ID) 

•  Need  to  have  front  station  threat  info.  To  increase  SA  and  decrease  interphone  chatter.  — 

Lives  depend  on  it. _ _ _ _ 
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9.  Rate  the  acceptability  of  Tanker  Rendezvous  (e.g.,  Inter-Leavened  Search  &  Track 
(ILST)  (Circle one)'. 

Average  Rating:  3.69  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  2 

b.  Very  Acceptable  1 

c.  Somewhat  Acceptable  1 

d.  Borderline  4 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  2 

g.  Totally  Unacceptable  3 

3  no  response 

Comments: _ 

•  Good  to  have  info 


•  ILST  -  nice 

•  Pilots  have  no  SA  as  to  the  benefits  of  ILST 

•  Unaware  of  ILST  changes  up  front 

•  No  difference/impact  to  pilots 

•  No  display  up  front 

•  need  to  repeat  information  up  front 

•  Problems  with  display  should  use  CAF  standard  for  display 

•  Cannot  rate  as  a  pilot.  Comm  from  OSO  was  great 

•  From  front  station  -  sounded  great 

•  Really  could  not  see  it  from  up-front  in  the  sim. 

•  Add  the  capability  to  the  front  station 

•  Great  capability  -  will  be  used  for  threat. 
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10.  Rank  Order  ( Low  =  i)  the  following  in  terms  of  Operational  Usefulness: 
For  each  write  in  what  Operation  Shortfall/Capability  it  addresses: 

R.O.  OPS  Issue 

Data  Link  _  _ _ _ 

•  18  hour  Global  Power  -  real-time  updates 

•  Package  integration/SA 

•  Absolute  need  for  SA  and  last  minute  changes 

•  Package  integration,  Theater  SA 

•  Time  critical  target  attack 

HUD  _  _ _ _ 

•  More  SA,  less  head  down  time 

•  Great  for  human  factors  and  clearing 

•  Takes  place  of  limited  VSD  and  add  visual  SA 

•  SA  and  targeting 

Color  Head  Down  Displays  _  _ _ 

•  MFD  would  allow  replacement  of  SMS  panel  RWR 

•  Required  moving  map  display 

•  More  threat/nav/mission/radar  information  in  flight 

•  Contrast,  more  intuitive  information,  CHEAPER! 

•  Common  front/aft  station  displays 

NVG  Lighting  _  _ _ _ 

•  Shortfall  -  ilium  stix  ineffective 

•  Current  system  has  severe  limitations 

•  Have  to  have,  mission  essential 

•  Makes  NVG  100%  useful  as  opposed  to  75% 

•  Today's  is  less  than  adequate 

•  Night  attack 

FLIR  _  _ _ _ 

•  Precision  weapons  capability/GBUs/LGBs 

•  PGM  capability 

•  Would  be  nice  to  have,  more  SA 

•  Whatever 

•  Precision  weapon  targeting 

Helmet  Mounted  Display  _  _ _ _ 

•  Nice,  but  not  mandatory 

•  Unless  you  get  the  weight  down  this  could  be  BAD.  NVGs  are  bad  enough 

•  SA  and  targeting 

Moving  Map  _  _ 

•  Improves  SA  dramatically 

•  Could  be  used  with  color  MFD 

•  LAR  display,  Digital  bullseye,  Datalink  display  all  covered 

•  Lack  of  RTIC/displays 


•  Everyone's  SA  up  50%  or  more 

•  SA  for  all  missions 

Off-Board  Mission  Planning  _  _ 

•  Already  exists 

•  Good  for  getting  changes  to  crew  on  long  missions 

•  All  attack  missions 

On-Board  Mission  Planning  _  _ 

•  Must  have  with  data  link  so  we  can  actually  use  information 

•  Re-tasking 

•  Second  best  to  off-board  mission  planning. 

•  Time  critical  target  attack 

Global  Air  Traffic 

Management  (GATM)  _  _ 

•  Soon  to  be  mandatory  for  certain  nav  tracks 

•  I  don't  even  know  what  this  is.  I  hear  Kosovo  was  cluttered  in  the  sky 

•  ? 

•  Global  nav. 

Digital  Bullseye  _  _ 

•  Need  immediate  B/E  for  large  force  execution 

•  With  moving  map 

•  With  pilot  repeater  scopes  below  airspeed  for  pilot  and  below  altimeter  for  copilot 

•  Threat  awareness  /  composite  force  package 

•  Pilot  SA  up  100% 

•  Lifeline  (common  ref.)  for  strike  packages. 

TSF  Compare  »»>  _  _ _ 

Other  _  _  _ 

•  Low  Light  TV 

•  RWR 

•  4th  generator 

•  RAW  repeater  for  pilots 

•  Engines  -  for  higher  (40,000+  Alt)  capabilities 
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1 1 .  After  the  B-1B  Block  D,  E,  and  F  Upgrades  are  completed,  what  will  be  the  Number 
1  Operational  Flight  Shortfall  of  the  Aircraft  (Ustp _ 

•  Cockpit  displays  and  computing  power.  With  better  displays  and  computers  (color  MFDs)  the 
capabilities  to  get  moving  map,  see  displays  at  other  station,  etc.  will  greatly  enhance  non¬ 
verbal  comm  and  SA. 

•  Real  time  target  information/threat  information 

•  To  be  able  to  data  link  and  share  information  with  other  package  players.  Rather  that 
piecemeal  and  Band-Aid  fix  the  cockpit  problems,  the  follow-on  package  should  completely 
re-vamp  the  cockpit  avionics.  Especially  if  the  B-1  will  be  around  until  2030  as  is  projected 

•  Spare  parts!  Being  able  to  load  flight  plan  and  CDC  flight  plan  from  the  seam  DTUC  rather 
than  two  separate  DTUCs. 

•  Moving  map, 

•  NVG  lighting, 

•  LAR  presentation 

•  \JSOW,  JASM  priority 

•  Outdated  computers 

•  New  cockpit  up  front 

•  Moving  map. 

•  Non-integrated  SA  -  i.e.  continue  to  have  to  use  interphone  for  everything 

•  IFF  -  capability  to  identify  friend  or  foe  in  an  allied  environment  that  includes  Russian  and 
French  aircraft.  This  would  assist  in  avoiding  only  enemy  aircraft 

•  Altitude  limitation  -  low  30s  is  still  accessible  to  many  threats.  Flying  above  FL400  would  be 
great. 

•  NVG  capability 

•  Computer  processing  time  needs  to  be  improved  to  1999/2000  standards  with  more  memory 

•  4th  AC  generator  for  improved  EMUX  reliability 

•  Icing/WX/IMC  flying 

•  Pilots  (and  WSOs)  need  a  color  video  display  depicting  the  LAR.  Something  like  this  (see 
drawing  below).  This  would  be  good  for  SA,  tell  us  how  to  get  to  the  TGT,  the  LAR,  and  avoid 
threats.  Of  course  update  every  few  seconds  at  least.  I  hope  this  does  not  cause  any  WSO 
union  legal  action. 

•  Data  link  (see  question  4).  Digital  Bull  and  BRAA  should  be  in  Block  D.  Participant  also  had 
drawing  (see  below) 
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•  Engines  limiting  (fuel  scheduling)  our  high  altitude  capability.  The  B-1  is  the  most  expensive 
theater  (minus  the  B-52)  for  dropping  bombs.  At  out  altitudes  we  require  package  protection 
and  air  refueling  (in  operation  Allied  Force.)  B-52s  flew  40,000+  MSL  and  didn't  need 
package  protection  or  air  refueling. 

•  Data  link 

•  Precision  weapon  targeting. 


B.  WSO  Experience:  (6  participants  Hew  as  DSO/OSO;  5  DSO  only;  5  OSO  only) 


Current  Qualified  Aircraft:  B-1B  Number  of  Flying  Hrs:  Ave:  966.38 
Range:  250-2250 

B-1B  DSO  Hrs:  Ave:  460.67  Ranee:  125-1125  A/C  Block:  B/C/D 
B-1B  OSO  Hrs:  Ave:  515.13  Range:  125-1250  A/C  Block:  B/C/D 


Other  Aircraft  Flown: _ Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _ Number  of  Flying  Hrs _ 

Other  Aircraft  Flown:  _ _ Number  of  Flying  Hrs _ 

Other  Aircraft  Flown: _ Number  of  Flying  Hrs _ 

Total  Flying  Hrs:  Ave:  1853.63  Range:  531-3750 

2.  B-1B  Simulator  Time:  Yes:  16_  No:  0 

Type  Sim:  WST.  CPT.  Integrated  Number  of  Hrs:  Ave:  115,67 
Range:  30-200 

Purpose:  (e.g.  Upgrade  training  for  Block  X):  All  blocks:  Student  training/ 
mission  planning:  Block  C  training:  training:  FTP;  IOC/CT;  Continuation/currency; 
Requirements 


4.  Have  you  had  previous  B-1B  Block  D,  E  and/or  F  experience?  Yes:  5  No:  1 1 

If  yes,  what  aircraft/simulator(s)?  4  A/C:  1  sim  Number  of  Hrs:  Ave:  69.5 

Range:  8-150 

5.  Have  you  had  any  AFMSS  Training?  Yes:  7  No:  9 

6.  Have  you  had  any  training  on  GMTI?  Yes:  5  No:  H 

7.  Have  you  had  any  training  on  JDAM?  Yes:  8  No:  8 

7.  Are  you  familiar  with  Launch  Acceptability  Regions  (LARs)?  Yes:  9  No:  7 

8.  a.  Have  you  ever  participated  in  any  B-1B  studies  or  flight  tests?  Yes:  6  No:  8 

b.  If  yes,  describe: 

•AFMSS  mission  prep; 

•Edwards  AFB  test  support; 

•FCF; 
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•MRT; 

•DMV-II; 

•Link  16; 

•FOTD 

•ALE50/55  Human  Factors, 
•Workload  and  SA  in  Link  16, 
•ALE-50  flights  at  Eglin 


DSO  Section 

1.  Rate  the  acceptability  of  the  modified  ASQ-184  RWR  Controls  &  Displays  (Circle): 

Average  Rating:  5.27  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  1 

b.  Very  Acceptable  2 

c.  Somewhat  Acceptable  7 

d.  Borderline  1 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments:  _ _ _ _ 

•  Interference  of  trunk  line  with  cardinal  directions  and  TWF  box 

•  Compass  Rose  and  course  lines  interfere  with  TWF  box 

•  Very  good  BRA  and  Bull  info;  nice  if  you  can  hook  and  emitter  of  the  TSF  and  the  cursor 
would  track  on  the  threat  (this  would  allow  you  to  track  a  threat  and  get  a  continuous  BRA 
and  Bull  update  for  that  threat.) 

•  More  interactive  capability  is  necessary 

•  No  frequency  readout  on  TSF  or  PF  display 

•  WSOs  always  need  to  be  able  to  change  techniques 

•  Need  NEAJAM!  Need  active  selection  of  other  techniques 

•  TSF  super  -  good  having  track  lines,  however  highlighting  the  portion  we  are  sequenced  to 
would  be  better.  Great  digital  bullseye/BRAA 

•  The  displays  represented  a  very  accurate  "DF"  capability  -  it  might  differ  in  the  actual  jet. 


2.  Rate  the  acceptability  of  the  CDU  Controls  &  Displays  for  RF  Countermeasures 
(i.e.,  Towed  Decoy  &  Low  Band  Jammer)  (Circle)'. 

Average  Rating:  5.73  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  8 

c.  Somewhat  Acceptable  3 

d.  Borderline  0 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments: _ 

•  Not  intuitive  on  activating/deactivating  low  band  jammer 

•  Unsure  how  to  activate  low-band  jammer.  Towed  decoy  worked  very  well. 

•  On  [AA]  put  a  column  showing  if  jamming  is  occurring  and  what  the  technique  is  on  a  threat 

•  DSO  needs  the  ability  to  deploy  countermeasures  at  will.  System  jammed  threats  at  too 
great  a  distance.  Such  practice  would  only  highlight  the  aircraft 

•  Better  actual  presentation  when  it  is  jamming  vs.  pending _ 
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3 .  Rate  the  acceptability  of  the  ALR-56M  (Circle)'. 

Average  Rating:  5.64  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  8 

c.  Somewhat  Acceptable  2 

d.  Borderline  1 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments: _ _ _ _ _ 

•  Good  (retains  a  lot  of  the  old  ALQ-161  display).  Needs  to  have  more  interaction  with  the  DSO 
(i.e.  if  a  threat  comes  up  out  of  it's  lethal  range,  the  it  should  not  automatically  jam.  The  DSO 
should  have  the  option  to  inhibit  jamming  if  necessary). 

•  DSO  needs  to  be  able  to  select  and  deselect  appropriate  jamming  techniques 

•  Looks  good  in  the  sim 

•  More  stable  than  ALQ-161  system  (maybe  sim-ism) 

•  Really  need  audio  on  all  capabilities. 

•  If  it  actually  works  as  predicted  then  it  is  all  good. _ _ 
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4.  Rate  the  acceptability  of  the  Inter-Crew  Coordination  (Circle): 

Average  Rating:  5.64  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  1 

b.  Very  Acceptable  5 

c.  Somewhat  Acceptable  5 

d.  Borderline  0 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments:  _ 

•  Experienced  crew  able  to  detect  most  discrepancies.  More  prebrief  info  on  weapons  loadout 
would  end  cockpit  discussions. 

•  More  info  needed  to  brief  the  sim  before  getting  in  the  box  (i.e.  weapons  load,  targets,  etc.) 

•  The  BRA/Digital  Bullseye  enhanced  crew  coordination.  These  functions  are  essential  to  the 
mission 

•  KindofN/A 

•  Pilots  need  more  SA  cues  -  LAR  TTG  to  release  on  VSD  similar  to  Block  D  currently 
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5.  Rate  the  acceptability  of  Communications  (Circle  one)  l 

Average  Rating:  5.30  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  5 

c.  Somewhat  Acceptable  4 

d.  Borderline  .  0 

e.  Somewhat  Unacceptable  1 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 

1  no  response 

Comments: _ _ _ _ _ 

•  No  problems 


•  Use  of  hot  mic  adds  confusion 

•  N/A 

.  No  change 

«  More  comm  due  to  pilots  not  having  LAR  TTG  on  VSD 


6.  Rate  the  acceptability  of  the  EW  MFD  Formats  ( Circle  one)  l 

Average  Rating:  5.55  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  1 

b.  Very  Acceptable  4 

c.  Somewhat  Acceptable  6 

d.  Borderline  0 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments:  _ 

•  See  comment  on  item  1 

•  See  comment  for  Item  1 

•  Again,  change  [AA]  page  to  reflect  what  type  of  jamming  is  being  used  against  a  threat. 
Good  use  of  being  able  to  load  three  different  Bulls. 

•  The  type  of  jamming  being  used  needs  to  be  displayed  on  the  [AA]  page. 

•  Limited  time  with  format.  Could  not  back-up  RDU  on  MFD 

•  See  #1 

•  Can  we  get  color  screens  to  offer  ability  to  reproduce  ellipse/LARs _ 
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7.  Rate  the  acceptability  of  the  CDU  &  ASQ-184  Combination  {Circle  one)*. 

Average  Rating:  5.27  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  5 

c.  Somewhat  Acceptable  4 

d.  Borderline  2 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments: _ _____ _ — 

•  Overall,  an  improvement  over  current  system.  Need  threat  audio  to  detect  mode  changes 
for  proper  crew  reaction 

•  Good  options  built-in  and  easily  accessible  through  menu  formats. 

•  The  entire  system  needs  to  be  more  interactive 

•  Same  as  before 

•  Since  these  systems  are  virtually  independent  their  "combination"  is  not  really  applicable 

•  ILST  is  good  -  may  need  a  BWI  read-out  underneath  current  track  read  out _ 
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8.  Rate  the  acceptability  of  the  RFS/ECM  Panel  ( Circle )  • 

Average  Rating:  5.30  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  1 

b.  Very  Acceptable  2 

c.  Somewhat  Acceptable  6 

d.  Borderline  1 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 

1  no  response 

Comments: 


•  No  capability  by  the  DSO  except  ON/OFF.  Need  to  be  able  to  inhibit  jamming  by  switching 
ARM/SAFE  on  transmitter 

•  Did  not  work  well  in  the  simulator.  The  system  displayed  no  apparent  function, 

•  Change  inoperative  button  to  something  useful 

•  Need  NEAJAM 

•  Not  much  to  do  -  no  change  for  keys  that  are  used. 

•  Discuss  more  on  how  to  enter  Lethal  vs.  Norm  mode  -  not  all  full-up  system  represented. 
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9.  Rate  the  acceptability  of  the  overall  Threat  Situational  Awareness  (Circle): 

Average  Rating:  5.73  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  3 

b.  Very  Acceptable  3 

c.  Somewhat  Acceptable  4 

d.  Borderline  1 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments: _ _ _ _ _ _ _ 

•  See  comment  for  Item  7 

•  Great!  Only  thing  was  to  be  able  for  the  cursor  to  track  a  threat  so  you  can  get  continuous 
Bull  and  BRA  information  for  the  threat. 

•  DSO  needs  to  be  able  to  control  the  type  of  jamming  being  used. 

•  The  Digital  Bull  and  BRAA  with  moving  NSEW  (Compass  Rose?)  is  a  must!  Huge 
increase  in  SA.  Provides  accurate  and  timely  location  of  threats  enabling  that  information 
to  be  passed  to  all  aircraft  in  strike  package  and  AWACS  for  total  package  SA 

•  Could  add  status  i.e.  search/track  and  range  on  TSF  displays 

•  New  TSF  format  with  cardinal  directions  is  great!  ALR56  will  take  getting  used  to.  I  felt 
less  SA  in  threat  environment  as  for  knowing  if  threats  were  "covered"  and  with  what. 

(Yes,  I  don't  trust  IDECM  to  do  its  thing  without  adult  supervision.) 

•  If  the  system  works  as  depicted  it's  acceptable  -  the  “threat"  ping  is  somewhat  confusing  - 

need  to  have  separate  tones.  _ _ _ _ _ 
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10.  Rank  Order  (Low  =  i)  the  following  in  terms  of  Operational  Usefulness: 
For  each  write  in  what  Operation  Shortfall/Capability  it  addresses: 

R.O.  OPS  Issue 


Data  Link  _  _ _ 

•  Improve  situational  awareness  (SA) 

•  SA 

•  Allows  the  crew  to  know  the  battle  picture 

•  See  other  comments 

•  Gives  targeting  flexibility 

•  Package  integration 

•  Threat  data,  fighter  picture 

•  Very  useful!  Want  it! 

•  comm  /  replan  option 

Color  Head  Down  Displays  _  _ 

•  See  other  comments 

•  Better  SA  displays 

•  SA 

Moving  Map  _  _ 

•  Improve  SA 

•  Navigation  /  SA 

•  More  for  OSO  use  would  greatly  increase  OSO's  SA  on  LAR 

•  See  other  comments 

•  Need  real-time  depiction  of  LAR 

•  Need  this  ASAP  (with  visual  LARs  and  threat  overlays) 

•  Increased  SA 

•  SA  /  replan  option 


Off-Board  Mission  Planning  _  _ 

•  AFMSS  needs  significant  improvement 

•  Decrease  the  mission  planning  time 

•  See  other  comments 

•  Planning  mission 

On-Board  Mission  Planning  _  _ 

•  Capability  to  calculate  new  LAR  in-flight 

•  See  other  comments 

•  Re-targeting,  change  of  plans 

Compass  Rose  _  _ 

•  Simple  improvement  that  significantly  improves  SA 

•  SA 

•  Increases  DSO  SA 

•  See  other  comments 

•  Adds  SA 

•  Increase  SA,  survivability 


•  Better  SA,  increase  awareness  of  threat  position 

•  SA 


Digital  Bullseye  _  _ 

•  SA  /  Communication 

•  Builds  OSO’s  SA  on  threat  picture 

•  See  other  comments 

•  Adds  SA 

•  Increase  SA,  package  integration,  and  survivability! 

•  Better  SA,  better  comms  with  AWACS  and  Blue  Air 

•  SA 


Other _  _  _ 

•  FUR 

•  RTIC  (Real-time  Information  in  Cockpit  -  multiple  sources) 


11.  After  the  B-1B  Block  D,  E,  and  F  Upgrades  are  completed,  what  will  be  the  Number 
1  Operational  Defensive  Shortfall  of  the  Aircraft  (Ustfi _ _ 

•  Relay  of  real-time  threat  /  target  change  info 

•  Data  link 

•  Trying  to  calculate  a  new  LAR  for  contingencies  in  the  air  (moving  map  for  OSO  with  updated 
LAR.  information.) 

•  Inability  to  easily  reselect  JDAMs  to  different  targets  while  airborne. 

•  The  OSO  is  very  busy  cycling  between  formats  (i.e.  LAR  location,  SMS  page,  CNAV 
summary)  during  the  bomb  run.  Combine  more  information  on  the  same  page 

•  The  DSO  will  not  be  able  to  jam  any  threats.  Should  the  system  make  a  mistake,  there  is  no 
way  to  correct  it.  The  only  apparent  way  to  stop  the  system  is  to  turn  it  off,  thus  leaving  no 
ECM  available.  The  DSO  should  have  the  ability  to  jam  individual  threats.  The  BRA/Digital 
bullseye  information  is  a  must  for  modern  warfare.  Other  systems  should  be  sacrificed  to  add 
this  capability  to  the  B-1B.  There  is  no  excuse  for  not  having  a  digital  bullseye. 

•  Inability  to  re-ID  threat 

•  Inability  to  reprogram  jam  technique 

•  Operator  inputs 

•  Let's  see  how  D,E  and  F,  really  work  when  fielded 

•  Need  repeater  for  pilot  station 

•  RTIC  Of  not  installed) 

•  Lack  of  self-protection;  air-to-air,  air-to-ground  (HARM)  ordinance 

•  Need  multiple  ECM  techniques  in  combination 

•  1122  taken  out  (need) 

•  NEAJAM  taken  out  (need)  -  Put  back  in. 

•  Kinematic  flares  /  multi-color  flares 

•  Higher  frequency  coverage  for  161/1 84 

•  RWR  display  in  front  cockpit 

•  Use  ATRJ  instead  of  ASPJ  for  jamming  processor 

•  Better  inlets;  more  speed 

•  Bigger  windows  for  DSO/OSO 

•  More  Gs 

•  For  defensive  system  I  think  the  biggest  shortfall  will  be  data  link  capability. 

•  Need  BULL  indications 

•  LAR  SA  indications  to  pilots _ 
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OSO  Section 


1.  Rate  the  acceptability  of  the  WCMD  display  formats  (Circle 

Average  Rating:  3.88  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  0 

c.  Somewhat  Acceptable  2 

d.  Borderline  4 

e.  Somewhat  Unacceptable  1 

f.  Very  Unacceptable  1 

g.  Totally  Unacceptable  0 

3  no  response 

Comments:  _ _ _ 


•  Confusion  on  procedures  to  effect  weapons  release 

•  N/A  (didn't  really  see  it) 

•  Once  all  parameters  are  met  except  reaching  launch  point  bomb  light  should  illuminate 
inbound  target  from  IP  (i.e.  switches  set,  aircraft  heading  within  tolerances 

•  N/A  for  mission  1 

•  Need  distance  to  LARI  Too  many  page  changes.  Information  overload;  after  some  practice, 
all  need  information  could  be  reduced  or  simplified  and  placed  on  a  "C"  navigation  summary 
page! 

•  Did  not  like  the  word  "NONE"  referring  to  weapons  not  allocated  to  target,  would  rather  see 
the  weapon's  name  there  such  as  CBU-103,  JDAM,  etc.  Let's  OSO  know  that  there  is  a 
weapon  at  that  station  verses  stating  none.  The  "X"  is  OK  for  a  released  weapon. 

•  Still  needs  some  kind  of  steering  to  LAR  _ _ _ 


2.  Rate  the  acceptability  of  the  JDAM  display  formats  (Circle): 

Average  Rating:  3.90  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  0 

c.  Somewhat  Acceptable  4 

d.  Borderline  2 

e.  Somewhat  Unacceptable  3 

f.  Very  Unacceptable  1 

g.  Totally  Unacceptable  0 

1  no  response 

Comments: 


•  LAR  display  did  not  seem  to  show  correct  information.  IR/IZ  displayed  immediately  on 
sequence  to  target,  but  weapons  did  not  release. 

•  The  formats  seem  ok,  but  they  are  located  in  too  many  different  locations.  For  example, 
maybe  combine  more  information  on  LAR  STAT  page 

•  Weapons  bays  need  to  be  labeled  on  [D]  page. 

•  Current  TTG  to  LAR  on  E  page  should  be  displayed  well  prior  to  IP.  Range  to  acceptable 
launch  point  should  be  displayed.  Easy  target  allocation  should  be  done  by  selecting 
location  and  station  -  "enter" 

•  TTG  to  release  should  be  displayed  on  the  LAR  page  for  each  LAR 

•  Takes  a  while  to  step  through  each  weapons  allocation  to  get  to  the  one  you  want.  Would 
be  nice  to  be  able  to  page  through  instead  of  just  stepping  through  each  individual 
weapon  allocation  to  get  where  you  want 

•  Blocked  status  of  JDAMs  should  be  reflected  on  E-format  not  just  on  D-format _ 
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3 .  Rate  the  acceptability  of  the  ILST  (Circle): 

Average  Rating:  6.91  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  10 

c.  Somewhat  Acceptable  1 

d.  Borderline  0 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments: _  , 

•  Significantly  improves  SA,  ensure  ease  of  switching  radar  modes  (ILST,  GMTI,  real  beam, 
rendezvous.)  A  few  display  indicators  caused  confusion  (e.g.  tanker  offset  symbology. 

•  Tanker  offset  information?  What  does  it  really  mean? 

•  More  usefulness  in  detecting  Al  threats  than  a  Tanker  rendezvous 

•  The  display  needs  to  be  less  cluttered.  A  current  target  often  overlaps  with  a  new  target. 
There  needs  to  be  a  way  to  separate  the  two. 

•  Optimally  -  track  more  than  one  target  (contact).  Move  radar  return  numeric  data  away 
from  actual  return 

•  If  using  for  tanker  rendezvous,  doesn't  show  beacon  information  for  the  tanker  requiring 
crew  coordination  to  confirm  correct  tracker. 

•  Nice  system!  But  MAKE  SURE  the  display  of  date  is  identical  to  all  other  aircraft  air-to-air 
radars.  The  is  an  AF  standard  for  data  display 

•  Too  much  written  information  on  actual  screen!  Need  center  azimuth  line. 

•  Good  SA  to  tanker 

•  Real  important  to  put  "Bullseye"  info  on  this  display _ _ _ 
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4.  Rate  the  acceptability  of  the  LAR  displays  and  targeting  information  (Circle  one)  l 

Average  Rating:  3.27  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  0 

c.  Somewhat  Acceptable  1 

d.  Borderline  5 

e.  Somewhat  Unacceptable  2 

f.  Very  Unacceptable  2 

g.  Totally  Unacceptable  1 

1  no  response 

Comments: 


•  Displayed  WCMD  LAR  fine  for  1st  target,  unable  to  display  LAR  for  2nd  target  complex 

•  See  comments  for  Item  2. 

•  Good  use  on  putting  the  ALR  location  on  [EB],  but  put  more  information  on  it.  Good  also 
to  put  more  target  information  on  the  [EB]  besides  the  bars  (i.e.  TTG  to  LAR). 

•  The  LAR  display  needs  to  be  re-vamped.  The  [EB]  page  is  nearly  useless.  There  is  no 
way  to  view  the  distance  between  the  individual  LARs.  There  needs  to  be  a  constant 
running  bearing  to  the  LAR,  regardless  of  whether  the  plane  is  sequenced  to  the  target  or 
not. 

•  OSO  needs  range  to  launch  point  (LAR)  suggestion:  Have  top  bar  of  LAR  bar  not  appear 
until  aircraft  is  within  all  parameters  to  launch  JDAM 

•  When  dropping  CBUs  countdown  to  weapons  release  did  not  work  cycled  between  59:59 
-  0:00  until  just  released  weapon  without  warning. 

•  Not  enough  SA  on  LAR  location 

•  Need  continuous  RANGE  and  TIME  information!  Need  center  azimuth  line!  Less 
information! 

•  Pilots  do  not  have  SA  on  LAR.  OSO  had  to  notify  pilots...  due  to  tasks  pilots  need 
information  displayed  on  LAD  on  VSD. 

•  No  LAR  steering,  plus  no  way  of  knowing  range  between  LARs _ 
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5.  Were  you  able  to  use  the  LAR  Display  to  steer  to  the  upcoming  weapons  LAR(s)? 

Yes  8  No  3  Comments: _ _ _ 

•  Not  required  based  on  aircraft  heading  but  could've  been  accomplished.  That  said,  the 
LAR  bar  depiction  does  not  give  the  magnitude  of  the  turn  required  to  meet  LAR 
parameters  (i.e.,  #  of  degrees  to  turn  to  meet  LAR  criteria) 

•  Not  effective  because  not  familiar  with  system 

•  The  LAR  display  provided  virtually  no  help  in  getting  to  the  LAR. 

•  During  the  sim  the  crew  steered  toward  LAR,  but  did  not  get  a  release  when  within  the  LAR 

•  Would  be  better  to  have  actual  bearing  in  degrees  instead  of  just  an  - ► 

•  Would  be  nice  to  know  where  LAR  centers  are  (for  better  steering  information  and  also 
maximum  stand  off) 

•  Yeah,  but  really  did  not  need  it.  Good  mission  planning/study  and  a  good  chart  would 
provide  same  information. 

•  However,  the  biggest  problem  is  having  to  switch  Format  Pages  too  often  during  bombing. 
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6.  Were  you  able  to  follow  the  sequence  of  weapons  releases  using  the  LAR  Display? 

Yes  4  No  6  Comments:  1  Yes/No _ 

•  Only  first  three  weapons  were  displayed  for  first  target  complex.  Only  1  weapon  displayed 
for  3rd  target  complex 

•  Lack  of  training  on  Blocks  made  it  difficult,  but  with  time  and  practice,  should  be  ok 

•  very  confusing  when  dropping  CBUs;  ok  with  JDAM 

•  I  understand  the  basics,  but  need  training  on  the  new  information. 

•  Too  busy  _ 
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7.  Rate  the  acceptability  of  the  Inter-Crew  Coordination  (Circle  onel 

Average  Rating:  5.09  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  1 

b.  Very  Acceptable  1 

c.  Somewhat  Acceptable  7 

d.  Borderline  2 

e.  Somewhat  Unacceptable  0 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 


Comments: _ _ _ 

•  Pilots  seem  to  have  no  information  on  the  LAR 

•  Weapons  status  needs  to  be  better  coordinated  between  OSO  and  the 
pilots.  The  door  configurations  differed  between  the  forward  and  aft 
station. 

•  CDU  and  INS  NAV  will  an  issue  to  work  through 

•  NUC  circuitry  is  another  switch  error  nightmare.  3  switches  to  get  the 
bombs  off 

•  WSOs  don’t  know  what  the  CNMS  is  providing  to  the  pilots 

•  Pilots  need  better  cueing  on  LAR  in  front  station  as  well  as  RAW  threat 
picture  with  ALR-56M  radar  scope. 

•  Better  controls  and  displays  up-front  would  minimize  this. _ 


8.  Rate  the  acceptability  of  the  Communication  &  Coordination  with  the  Outside 

World  (Circle one)’. 

Average  Rating:  4.90  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  2 

c.  Somewhat  Acceptable  6 

d.  Borderline  1 

e.  Somewhat  Unacceptable  1 

f.  Very  Unacceptable  0 

g.  Totally  Unacceptable  0 

1  no  response 

Comments: 


•  No  "background"  communications  which  would  detract  from  SA  or  help. 

•  Need  data  link. 

•  The  radios  are  not  enough  to  properly  communicate.  A  data  link  is  essential 

•  No  hangers(??)  in  aft  station 

•  N/A 


72 


9.  Rate  the  acceptability  of  the  Target  Situational  Awareness  (Circle)'. 

Average  Rating:  4.09  (Highest  score  -  7;  lowest  1) 


a.  Totally  Acceptable  0 

b.  Very  Acceptable  1 

c.  Somewhat  Acceptable  4 

d.  Borderline  2 

e.  Somewhat  Unacceptable  3 

f.  Very  Unacceptable  1 

g.  Totally  Unacceptable  0 


Comments: _ _ _ _ _ _ 

•  LAR  did  not  help  awareness  of  where  the  release  should  occur. 

•  Once  a  target  has  been  hit,  have  the  current  target  information  come  up  on  the  [FCC]  first 
and  also  on  the  [E]  page.  For  example,  had  to  use  my  arrow  keys  to  see  target  10 
information  after  we  passed  over  target  8. 

•  Need  targets  depicted  on  chart,  without  range  information  to  acceptable  launch  point  is 
imperative. 

•  Need  to  be  able  to  keep  TTG  to  release  up  at  all  time  during  IP  to  target  run. 

•  need  to  get  familiar  with  LARs 

•  Bombs  start  coming  off  sooner  than  expected.  Lack  of  SA 

•  New  systems  are  not  responsible  for  this 

•  Displays  for  JDAM  and  WCMD  are  too  busy.  Also  OSO  had  to  change  to  different  formats 
too  often. 

•  Our  mission  planning  was  unrealistic  -  better  prior  mission  planning  would  eliminate  some 

of  this  confusion _ _ _ _ _ 
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10.  Rank  Order  (low  =  i)  the  following  in  terms  of  Operational  Usefulness: 
For  each  write  in  what  Operation  Shortfall/Capability  it  addresses: 


R.O. 


OPS  Issue 


Data  Link 

•  SA 

•  allows  the  crew  to  know  the  battle  picture 

•  mission  essential 

•  improvement  of  time  management 

•  Target  changes  /CAS/AI 

Color  Head  Down  Displays  _  _ 

•  Important  for  information  sorting 

•  More  efficient  cockpit  management 

Moving  Map  _  _ 

•  More  for  OSO  use  would  greatly  increase  OSO's  SA  on  LAR 

•  Essential  for  LAR  usage 

•  (Must  have);  most  important  for  SA 

•  Huge  SA  builder 

•  SA  greatly  needed 

•  Don't  ask-absurd  that  a  $280  million  airplane  doesn't  have  it!! 

Off-Board  Mission  Planning  _  _ 

•  Decrease  the  mission  planning  time 

•  Need  the  PC  PGM  solution  ASAP  (kill  MPS) 

•  AFMSS  -  giant  paperweight 

On-Board  Mission  Planning  _  _ 

•  Capability  to  calculate  new  LAR  in-flight 

•  Should  be  last  consideration 

•  With  data  link  capability  re-targeting  will  be  routine  and  this  is  needed 

•  On  global  power  mission,  a  must 

•  I'm  a  WSO  I  can  do  it  now 

Global  Air  Traffic 

Management  (GATM)  _  _ 

•  Increases  DSO  SA 

•  ? 

Digital  Bullseye  _  _ _ 

•  Builds  OSO's  SA  on  threat  picture 

•  Virtually  impossible  to  function  in  modem  aircraft  without  it. 

•  A  must  for  package  integration  and  survivability 

•  Too  easy,  make  it  happen 

Other _ 

•  FLIR 


RTIC;  threats 

Need  new  MFDs  and  graphics  generator.  Need  new  color  MFDs  and  true  graphics 
generators  to  allow  displayed  information  on  weapons  to  be  better  depicted. 


1 1 .  After  the  B-1B  Block  D,  E,  and  F  Upgrades  are  completed,  what  will  be  the  Number 
1  Operational  Offensive  Shortfall  of  the  Aircraft  (List)? 


•  See  previous  comments  in  OSO  section 

•  Data  link 

•  Trying  to  calculate  a  new  LAR  for  contingencies  in  the  air  (moving  map  for  OSO  with  updated 
LAR  information. 

•  Inability  to  easily  reselect  JDAMs  to  different  targets  while  airborne. 

•  The  OSO  is  very  busy  cycling  between  formats  (i.e.  LAR  location,  SMS  page,  CNAV 
summary)  during  the  bomb  run.  Combine  more  information  on  the  same  page 

•  In  order  to  properly  utilize  JDAM,  JCEM  and  JSOW  a  moving  map  display  must  be  used. 
Proper  implementation  of  these  systems  will  not  occur  until  a  moving  map  display  is  integrated 
into  the  airframe.  The  aft  avionics  should  be  stripped  and  replaced.  The  lack  of  computing 
power  is  unacceptable.  The  current  upgrades  are  good,  but  they  cannot  make  up  for  the 
necessary  hardware  changes.  Most  of  the  displays  and  computers  need  to  be  replaced.  A 
FLIR  would  add  a  lot  to  the  aircraft. 

•  ACUC  future  limitations  for  follow-on  weapons  and  upgrades.  "Avionics  computers  need  to  be 
Y2K  technology." 

•  Need  to  wait  and  see. 

•  Beyond  line-of-sight  data  link 

•  Two-way  communications,  Information  passing 

•  4th  generator 

•  Computers 

•  RWR  repeater  display  for  pilots 

•  RTIC 

•  Flare  button  for  pilots 

•  Replacement  parts  (parts  train) 

•  Need  the  ability  to  precision  guide  weapons  via  data  link,  TV-guided  weapons  and  laser- 
guided  systems  incorporated  on  the  B-1 

•  Engine  performance  -  need  new  engines. 

•  -  Better  controls  and  display  for  both  front  and  back  station  to  help  our  SA _ 
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APPENDIX  3  -  GENERAL  ORIENTATION  BRIEFING 


B-1B  BASELINE  STUDY 
ORIENTATION 


Kltam  G.  Kjknw 
785-4258 
mtKtat 
785-78X2 
ScoaHrovoa 
785-8290 
EffFC-CSEF 

1 _ 


Airport  to  WPAFB 


AreaB 


Baseline  Study 
Objective 


•  Examine  Effects  of  Blocks  D,  E,  &  F  on  Crew 
Woridoad,  Situational  Awareness,  &  Coordination 

•  Establish  Measurable  Standards  to  Judge  Future 
Block  Upgrades 


Baseline  Study 
Particulars 

•  8  Full  B-l  B  Crews  will  participate 

•  B-l  B  ERS  Baseline  Study  Configuration 

-  Pilct/Copilot  =  Block  D 

-  DSO  =  Block  F 

-  OSO  =  Block  E 

•  Data  Will  be  Recorded  &  Analyzed 

-  Subjective  Workload  Avesament  Technique  (SWAT) 

-  Situational  Awareness 

-  QueaRnnaires 

-  Video  Tape 

•  3  Practice  Missions  &  2  Data  Missions 

•  Report  will  be  issued  to  ASC/YDE 


SUBJECTIVE  WORKLOAD 
ASSESSMENT  TECHNIQUE 
(SWAT) 


SWAT 

Time  Load 

*  Total  Amount  of  Time  available  to  accomplish  a 

•  Workload  is  Defined  as: 

-  Time  Load 

task  as  well  as  overlap  of  tasks  or  parts  of  tasks 

-  Mental  Effort  Load 

1 .  Often  have  spare  time  (Low) 

-  Psychological  Stress  Load 

2.  Occasionally  have  spare  time  (Medium) 

3.  Almost  never  have  spare  time  (High) 

Mental  Effort  Load 

Amount  of  Attention  or  Concentration  that  is 
required  to  perform  a  task 

1 .  Very  little  conscious  mental  effort  or  concentration 
required  (Low) 

2.  Moderate  mental  effort  of  concentration  required 
(Medium) 

3.  Extensive  mental  effort  and  concentration  are 
necessary  (High) 


Psychological  Stress  Load 

The  Presence  of  Confusion,  Frustration,  and/or 
Anxiety  associated  with  task  performance 

1 .  Little  confusion,  risk,  frustration,  or  anxiety  exists  and 
can  be  easily  accommodated  (Low) 

2.  Moderate  stress  due  to  confusion, frustration,  or  anxiety 
noticeably  adds  to  workload  (Medium) 

3.  High  to  very  intense  stress  due  to  confusion, 
frustration,  or  anxiety  (High) 
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SWAT 

Card  Sorting 

Done  Once  Prior  to  any  Missions 

Arrange  27  Cards  from  Low  Workload  to  High 

Each  Card  has  3  Descriptors: 

-  Time  Load 

-  Mental  Effort  Load 

-  Psychological  Stie«  Load 

Each  with  Assigned  Level 

-  1  (Low),  2  (Medium)  or  3  (High) 

Card  Sort  is  for  SWAT  Familiarization 
Card  Sort  Aids  SWAT  Analysis 


SWAT 

Event  Scoring 

•  Asked  for  during  4  Mission  “Freezes’* 

•  Each  Crew  Member  will  Assign  either  a  1,  2,  or  3 
to  each  of  the  3  Categories: 

-  Time  Load 

-  Mental  Effort  Load 

-  Psychological  Stress  Load 

•  Category  Order  will  Always  be  the  Same 

•  (SA  will  also  be  collected  during  same  ' *  freeze  ” 
just  after  SWAT) 


Situation  Awareness  (SA) 


Situational  Awareness 

•  Situational  Awareness  (SA)  is  crew  member’s 
internal  model  of  the  world  around  him/her  at  any 
point  in  time 

-  Level  1 :  Perception  of  Elements  in  Current  Situation 

(fcg..  Red  Light  Appars) 

-  Level  2:  Comprehension  of  Current  Situation 

(e.g.,  Red  Light  mom  Engine  #4  is  on  Fire) 

-  Level  3:  Projection  of  Future  Status 

(eg.,  PuriMTg  Fire  Bctinguirfier  Button  will  put  out  Fire) 


Situational  Awareness 

•  Crew  flies  mission 

•  4  Mission  “Freezes” 

•  Crew  is  individually  questioned  about  SA 

-  Own  ship  and  Environment 

-  SA  Questions  Selected  Randomly  from  List 

-  (SWAT  will  also  be  collected  during  same  “freeze"  just 
before  SA) 

•  Answers  are  compared  to  Actual  Situation 

•  SA  score  is  determined  (SA/No  SA) 


Situational  Awareness 
Pilot  &  Copilot 

•  Own  Ship 

-  Heading  (+/-3°> 

-  Attitude  (+/-300  8.) 

-  Radio  Frequency  (+/-  0.50) 

•  Environment 

-  Threat  Azimuth  (+/- 5°) 

-  Threat  Type/Status  (Zero) 

-  Time  to  Waypoirt/Taiget  (+/-  10  seconds) 

-  Bearing/Heading  to  Targct/LAR  Envelope  (+/-  5°) 


Situational  Awareness  -  DSO 

Own  Ship 

-  ASQ-184  Mode  (RMP  ALTTO,  RMP  SEMI,  PREBRIEE,  RMP  MAN) 

-  ALR-50  SUtua/Mode  (Decoy  Statue,  Left/Right  Launcher, 
Tranitting.  AuUVMan,  Technique) 

-  ALR-56M  Mode  (Lethal/Nonna I) 

-  Lew  Baid  On-Board  Jammer  Status  (Gencnting  RF  [Rayi], 
Decided  Mode  [Brackets]) 

Environment 

-  Threat  Slatuj  (“New  Guy"  Fladi,  Priority  Di*ncnd  for  Float, 
Sequenced,  and  Select  Audio,  Fhdwig  Launch  Box,  Jammed/Nct 
Jammed  Emitter  [dim  cm  Battle  Mmagorat  Page] ) 

-  Thre*  Type  (SAM.  AAA,  MIG) 

-  Tire*  Priority  (1.2,  or  3) 

-  Threat  ALR-56M  Azimuth  (AQA  +/-  5°) 

-  Threat  ALR-56M  Range  (+/-  1  Ntn) 


Situational  Awareness  -  OSO 

•  Own  Ship 

-  WCMD/JD  AM  (Type/Location,  Status,  Target  SN,  Key  Status) 

-  Hung  Stores  (Bay  1,2, 3,  Inboard/Outboard) 

-  Bearing/Range  from  A/C  to  Cuisor  (BRA) 

-  ILST  Status/Mode 

-  Own  Ship  Location  (Lat/Lcng  fhoure.  minutes]) 

•  Environment 

-  Relative  LAR  Location  (Headng,  RightAeft  Ooaa  TVack  Range, 
JDAM  Conctrahed  ['*m-zme''yUncoratnined  [“irwai^e”),  WCMD  SFW 
“in-zone"  ^’in-range”) 

-  Time  to  Waypoint/Target  (+/- 10  recand*) 

-  Targets  (Tgta  Enabled/All.  Tgt  SN,  Type,  Bearing.  TTG,  Rangc/Zmc 
Statue,  Delivery  Statua  [AVL/RBQ]) 

-  New  Target  Location  (Ltf/Lang  [horn,  mbmles]) 


Time  and  Accuracy  Measures 

May  Be  Culled  from  Video  Tape 
Examples: 

Questionnaires 

•  Personnel  Information 

•  Ptfot/Copilot  -  Time  to  Recognize  &  Avoid  Threat  &  any  Errors 

•  Pilot/Copilot  -  Time  to  Enter  new  Radio  Frequency  &  any  Errors 

•  Experience 

•  Pflot/Copilot  -  Time  to  Regain  LAR  &  any  Eirors 

•  Ratings 

•  OSO  -  Time  to  Eider  New  Waypoint  Sc  any  Enors 

•  OSO  -  Time  to  Resequence  around  Hung  Stores  &  any  Errors 

•  Rank  Ordering 

•  OSO  -  Time  to  Divert  Sc  any  Errors 

•  DSO  -  Time  to  Deploy  ALR-50  Towed  Decoy  &  any  Errors 

•  DSO  -  Time  to  Recognize  Threat  &  any  Errors 

•  Comments 

SWAT 

Card  Sorting 

•  3  Factors  each  with  3  levels  -  27  Possible  Combinations 

•  27  Cards 

•  Each  Card  has  3  Factors  on  it 

•  Arrange  deck  from  Lowest  Workload  Card  (botto*  of  deck)  to 
Highest  Workload  Card  (top  of  deck) 

•  No  Wrong  Answer 

•  Arrangement  of  Deck  will  be  your  Personalized 
Prioritization  of  Workload 


,07.00 

Wl  06.50.00 


00:37:00 
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APPENDIX  5  -  MISSION  BRIEFING  SLIDES 


Missions 

General  Mission  Brief 

•  B-l  B  ERS  is  like  CPT  not  WST 

•  B-l  B  ERS  Take-Off  Procedure 

-  A/C  is  parked  on  Ground 

-  DSO  •‘Unfreezes"  Simulator 

-  Advance  Throttles 

-  Parking  Brake  OFF 

-  Taxi  to  Correct  Heading 

-  Use  Normal  Take -Off  Procedure 

•  Try  to  Operate  only  Necessary  Controls 

•  Auto-Pilot  not  Coupled  to  CMNS  (FLT  DIR  Rotary  Knob  portion 
“CNMS”) 

General  Mission  Brief 

General  Mission  Brief 

•  HIT  YOUR  ETA* 

•  B-l  B  ERS  “Notes”  are  on  Whiteboard 

♦  To  Start/Unfreeze  Mission,  DSO  presses  Recorder 

•  Leave  microphones  on  HOT  MIC 

ON/AUTO  on  IKB  Pnl  once  Everybody  is  Ready  (This  stun 

•  Use  True  Course  Heading  (No  mag  var  «  B«ob/N«v  a  cnms 

Mutator,  ETA  countdown.  computer*.  etc.) 

Readouts) 

•  Fly  at  about  24,000  MSL 

•  On  ILST,  Tanker  is  Assigned  25,000  ft  MSL 

•  No  Need  for  TAL  Maneuver  to  Align  JDAM/WCMD 

♦  Full  CHAFF/FLARE  Load 

Inertial 

•  TSF  will  have  Compass  Rose  &  Digital  Bulls  Eye 

•  JDAM/1  =  Mk  84,  JDAM/3  =  BLU-1 09 

•  Use  Expanded  Checklist,  Section  2,  for  JDAM  Info 

•  May  try  MAN  Jettison  or  Reallocate  Bombs  if  Hung  Store 
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Threats 

•  SA-3GOA 

•  SA-6  GAINFUL 

•  SA-8  GECKO 

•  DO  NOT  GO  BELOW  15,000  MSL 

-  ZSU-23-4  SHILKA 

-  Possible  SA-7s  &.  SA-9s 

•  Threat  Warning  Tones  Work  (dso  «*c  ^  vol 

ccMret  keob  ■  @  MAX  VOL  cm  later-CoM*  P«cl) 


Practice  Missions 

•  3  Practice  Missions 

1 .  ILST  -  KC-1 35  @  WP  3  w/Orbit  Pattern  20  x  20  miles 

2.  Threat  near  WP  5 

3.  Hung  JDAM,  Threats 

•  Common  Information 

-  All  3  Routes  are  the  Same 

-  WPs  1-8 

-  WP  1  is  T/O  Point 

-  WP  5  =  IP,  WP  6  =  TGT  (T«k  b.  HQ) 

-  Fwd  8  JDAM/3  (Delay  Fttee),  Int  8  JDAM/1  (Coatact  F»*e), 
Aft  1 0  WCMD  CBU-1 03  OEM  (proximity  fw) 

-  26  Total  Bombs 


Data  Missions 

Mission  1  -  UTTR  Scenario 

•  2  Types  of  JDAM*  (Fwd  -  8  BLU-)  09.  WDeky  F»wt  lot  -  8  Mk  44. 
w/Ccmua  Rut.  Afl  -  8  BLU  1 09*  w/Deky  Fuac)  (24  Total) 

•  Possible  Hung  Storefs)  fTiy  to  Use  Appropriate  Store*.  BLU-109OO 
Had,  Mk  S4  oa  Soft) 

•  WPs  4  -  15,  IPs  =  D7,  D9,  D1 1 

•  3  Target  Groups  (DS  CoaxiMd  A  Cootrol  Buakcr.  D10  Electrical 
Traarionaer  S«b-St«rioa,  DI2  Tuanel  Com  plot) 

•  Possible  Threat  Diversion  between  WP  D9  &  DIO  <sa-6. 

Notch  Rmgt  Eavtlopc  Clockwise  to  Ea«) 

•  Possible  Diversion  to  Alternative  Landing  Field  (a  Pilot 

eater  TACAN  code  ido  CDU,  b.  or  cater  aew  destkafioa  as  a  Way  Point  into 
flight  plM  ariag  3-letter  ID,  c.  orLAT/LONG:  OSO  enter  LAT/LONG  into 
Nav  Page) 


Data  Missions 

Mission  2  -  Powder  River  Scenario 

•  Tanker  Orbit  between  WP  D4  &  D5  (ilst) 

•  Don’t  have  TDS  deployed  during  TNKR 

RNDV S  (Tanker  is  A/C  @  25K  ft  MSL) 

•  WCMD  CBU- 103  CEM  (10  FwdBay  w/Proximity 

Fuse,  1 0  Int  Bay  w/Proxmnty  Fuse,  Aft  Bay  Empty)  (20  Total) 

•  Red  Circles  on  Map  MAY  be  Threats 

•  WPs  2  -  13,  IPs  =  D7,  D9,  D1 1 

•  3  T arget  Groups  p8  Truck  Conwy,  DIO  Troops  in  Open, 
DI2  Truck  Convoy) 
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APPENDIX  8  -  INDICES  OF  CENTRAL  TENDENCY  FOR  SWAT 


TABLE  1.  INDICES  OF  CENTRAL  TENDENCY  AND  DISPERSION  FOR  SWAT 

-CREW 


MSN 

1 

2 

FRZPT 

FRZPT 

1 

2 

3 

4 

Total 

1 

2 

3 

4 

Total 

SWAT 

Mean 

13.95 

19.71 

30.65 

31.03 

23.97 

8.03 

11.80 

30.56 

25.74 

19.03 

Median 

10.70 

38.50 

32.88 

21.90 

.00 

5.35 

29.93 

27.25 

14.30 

Mode 

.00 

.00 

38.50 

.00 

— 

.00 

.00 

.00 

IH&H 

.00 

Std  Deviation^ 

19.63 

25.41 

18.46 

28.15 

wzmM 

12.93 

15.00 

23.51 

Em 

21.12 

63.75 

loo.ool 

63.75 

100.00 

100.00 

49.15 

70.40 

70.40 

Minimum 

.00 

.00 

.00 

.00 

.00 

.00 

.00 

.00 

.00 

Maximum 

63.75 

100.00 

63.75 

100.00 

100.00 

49.15 

49.15 

70.40 

70.40 

70.40 

TABLE  2.  INDICES  OF  CENTRAL  TENDENCY  AND  DISPERSION  FOR  SWAT 

-PILOTS  (FLIGHT  CREW) 


MSN 

1 

2 

FRZPT 

FRZPT 

1 

2 

3 

4 

Total 

1 

2 

3 

4 

Total 

SWAT 

Mean 

9.58 

9.35 

29.67 

26.22 

19.01 

11.28 

15.76 

33.23 

29.68 

22.49 

Median 

.00 

.00 

38.50 

27.25 

16.55 

.00 

16.55 

Mode 

.00 

.00 

MM 

MM 

.00 

El 

EEXEI 

.00 

Std  Deviation 

16.17 

15.21 

IMMcl 

EEi 

IEE3 

e m 

US! 

22.68 

Range 

49.15 

49.15 

EE1 

[jJBl 

49.15 

70.40 

70.40 

70.40 

Minimum 

.00 

.00 

iil 

mm 

H 

.00 

.00 

.00 

.00 

Maximum 

49.15 

49.15 

49.15 

64.20 

64.20 

49.15 

49.15 

70.40 

70.40 

70.40 
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TABLE  3.  INDICES  OF  CENTRAL  TENDENCY  AND  DISPERSION  FOR 

SWAT-OSO 


MSN 

1 

2 

FRZPT 

Total 

FRZPT 

Total 

1 

2 

3 

4 

1 

2 

3 

4 

SWAT 

Mean 

40.45 

9.84 

9.85 

34.16 

27.88 

20.43 

Median 

feEi.nl 

37.50 

7.15 

7.15 

39.35 

29.80 

15.50 

Mode 

■HUM 

19.10 

48.90 

67.30 

48.90 

.00 

.00 

48.90 

15.50 

.00 

21.48 

27.66 

7.65 

30.60 

24.89 

22.86 

16.99 

19.28 

48.90 

80.90 

15.40 

100.00 

100.00 

67.00 

67.00 

Minimum 

.00 

19.10 

33.50 

.00 

.00 

.00 

.00 

.00 

Maximum 

48.90 

100.00 

48.90 

100.00 

34.60 

W1 

67.00 

48.90 

67.00 

TABLE  4.  INDICES  OF  CENTRAL  TENDENCY  AND  DISPERSION  FOR 

SWAT-DSO 


MSN 

1 

2 

FRZPT 

Total 

F 

RZPT 

Total 

1 

2 

3 

4 

1 

2 

3 

4 

SWAT 

14.05 

BEi 

liiaa 

18.34 

Pig 

ESDI 

EESQ 

11.59 

.00 

WEM 

.00 

1*1*1 

Pig 

KTHEZil 

WEHS! 

.00 

Mode 

.00 

.00 

.00 

MtUU 

Pin 

1il»B 

Std  Deviation 

23.01 

ESI 

Kim 

EB1 

HE3 

Range 

63.50 

EMfcM 

63.50 

K*I*I 

19.10 

48.90 

48.90 

48.90 

Minimum 

.00 

HI 

.00 

.00 

ESS 

.00 

.00 

.00 

.00 

Maximum 

63.50 

48.90 

63.50 

48.90 

63.50 

K»Itl 

19.10 

48.90 

48.90 

48.90 

TABLE  5.  CORRELATION  COEFFICENTS  -  CREW 


CREW 

MSN 

FRZPT 

SWAT 

CREW 

Pearson  Correlation 

1.000 

.000 

.000 

.212* 

Sig.  (2-tailed) 

1.000 

1.000 

.001 

N 

256 

256 

256 

252 

MSN 

Pearson  Correlation 

.000 

1.000 

.000 

-.109 

Sig.  (2-tailed) 

1.000 

1.000 

.084 

N 

256 

256 

256 

252 

FRZPT 

Pearson  Correlation 

.000 

.000 

1.000 

.333* 

Sig.  (2-tailed) 

1.000 

1.000 

.000 

N 

256 

256 

256 

252 

SWAT 

Pearson  Correlation 

.212* 

-.109 

.333* 

1.000 

Sig.  (2-tailed) 

.001 

.084 

.000 

. 

N 

252 

252 

252 

252 

*  Correlation  is  significant  at  the  0.01  level  (2-tailed) 
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TABLE  6.  CORRELATION  COEFFICIENTS  -  PILOTS 


CREWPOS 

MSN 

FRZPT 

SWAT 

CREWPOS 

Pearson  Correlation 

1.000 

.000 

.000 

-.060 

Sig.  (2-tailed) 

1.000 

1.000 

.504 

N 

128 

128 

128 

126 

MSN 

Pearson  Correlation 

.000 

1.000 

.000 

.083 

Sig.  (2-tailed) 

1.000 

1.000 

.355 

N 

128 

128 

128 

126 

FRZPT 

Pearson  Correlation 

.000 

.000 

1.000 

.372* 

Sig.  (2-tailed) 

1.000 

1.000 

. 

.000 

N 

128 

128 

128 

126 

SWAT 

Pearson  Correlation 

-.060 

.083 

.372* 

1.000 

Sig.  (2-tailed) 

.504 

.355 

.000 

N 

126 

126 

126 

126 

*  Correlation  is 

significant  at  the  0.0 1 

level  (2-tailed) 

TABLE  7.  CORRELATION  COEFFICIENTS  -  OSO 

CREWPOS 

MSN 

FRZPT 

SWAT 

CREWPOS 

Pearson  Correlation 

Sig.  (2-tailed) 

• 

N 

64 

64 

64 

63 

MSN 

Pearson  Correlation 

1.000 

.000 

-.416* 

Sig.  (2-tailed) 

1.000 

.001 

N 

64 

64 

64 

63 

FRZPT 

Pearson  Correlation 

.000 

1.000 

.383* 

Sig.  (2-tailed) 

1.000 

.002 

N 

64 

64 

64 

63 

SWAT 

Pearson  Correlation 

-.416* 

.383* 

1.000 

Sig.  (2-tailed) 

.001 

.002 

. 

N 

63 

63 

63 

63 

*  Correlation  is 

significant  at  the  0.01  level  (2 -tailed) 

TABLE  8.  CORRELATION  COEFFICIENTS  -  DSO 

CREWPOS 

MSN 

FRZPT 

SWAT 

CREWPOS 

Pearson  Correlation 

. 

Sig.  (2-tailed) 

• 

N 

64 

64 

64 

63 

MSN 

Pearson  Correlation 

1.000 

.000 

-.110 

Sig.  (2-tailed) 

1.000 

.391 

N 

64 

64 

64 

63 

FRZPT 

Pearson  Correlation 

.000 

1.000 

.258* 

Sig.  (2-tailed) 

1.000 

.042 

N 

64 

64 

64 

63 

SWAT 

Pearson  Correlation 

-.110 

.258* 

1.000 

Sig.  (2-tailed) 

.391 

.042 

. 

N 

63 

63 

63 

63 

*  Correlation  is  significant  at  the  0.05 

level  (2-tailed) 

APPENDIX  9  -  MANO VA  TABLES  FOR  SWAT  DATA 


TABLE  1.  REPEATED  MEASURES  MANOVA  -  MISSION  AND  FREEZE  POINT 
DEPENDENT  MEASURE:  SWAT 
CREW 


Effect 

Value 

F 

Hypothesis  df 

Error  df 

Sig. 

Eta  Squared 

MSN 

Pillai's  Trace 

.268 

1.099 

1.000 

3.000 

.371 

.268 

Wilks'  Lambda 

.732 

1.099 

1.000 

3.000 

.371 

.268 

Hotelling's 

Trace 

.366 

1.099 

1.000 

3.000 

.371 

.268 

Roy's  Largest 
Root 

.366 

1.099 

1.000 

3.000 

.371 

.268 

FRZPT 

Pillai's  Trace 

.969 

10.294 

3.000 

1.000 

.224 

.969 

Wilks'  Lambda 

.031 

10.294 

3.000 

1.000 

.224 

.969 

Hotelling's 

Trace 

30.88 

3 

10.294 

3.000 

1.000 

.224 

.969 

Roy's  Largest 
Root 

30.88 

3 

10.294 

3.000 

1.000 

.224 

.969 

MSN* 

FRZPT 

Pillai's  Trace 

1.000 

2676.20 

3 

3.000 

1.000 

.014 

1.000 

Wilks'  Lambda 

.000 

2676.20 

7 

3.000 

1.000 

.014 

1.000 

Hotelling's 

Trace 

8028. 

610 

J 

2676.20 

3 

3.000 

1.000 

.014 

1.000 

Roy's  Largest 
Root 

8028. 

610 

2676.20 

3 

3.000 

1.000 

.014 

1.000 

TABLE  2.  SIMPLE  MAIN  EFFECT  MANOVA:  FREEZE  POINT 

MISSION  1 
CREW 

Multivariate  Tests 


Effect 

Value 

F 

Hypothesis  df 

Error  df 

Sig. 

FRZPT 

Pillai's  Trace 

.795 

5.177 

3.000 

4.000 

.073 

Wilks’  Lambda 

.205 

5.177 

3.000 

4.000 

.073 

Hotelling's  Trace 

3.883 

5.177 

3.000 

4.000 

.073 

Roy's  Largest  Root 

3.883 

5.177 

3.000 

4.000 

.073 

TABLE  3.  SIMPLE  MAIN  EFFECT  MANOVA:  FREEZE  POINT 

MISSION  2 
CREW 

Multivariate  Tests 


Effect 

Value 

F 

Hypothesis  df 

Error  df 

Sig. 

FRZPT 

Pillai's  Trace 

.748 

4.958 

3.000 

5.000 

.059 

Wilks'  Lambda 

.252 

4.958 

3.000 

5.000 

.059 

Hotelling's  Trace 

2.975 

4.958 

3.000 

5.000 

.059 

Roy's  Largest  Root 

2.975 

4.958 

3.000 

5.000 

.059 
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TABLE  4.  REPEATED  MEASURES  MANOVA  -  MISSION  AND  FREEZE  POINT 
DEPENDENT  MEASURE:  SWAT 
PILOTS  (FLIGHT  CREW) 


Multivariate  Tests 


Effect 

Value 

F 

Hypothesis  df  Error  df 

Sig. 

MSN 

Pillai's  Trace 

.016 

.048 

1.000 

3.000 

.840 

Wilks'  Lambda 

.984 

.048 

1.000 

3.000 

.840 

Hotelling's  Trace 

.016 

.048 

1.000 

3.000 

.840 

Roy's  Largest  Root 

.016 

.048 

1.000 

3.000 

.840 

FRZPT  Pillai’s  Trace 

.969 

10.503 

3.000 

1.000 

.222 

Wilks'  Lambda 

.031 

10.503 

3.000 

1.000 

.222 

Hotelling's  Trace 

31.508 

10.503 

3.000 

1.000 

.222 

Roy’s  Largest  Root 

31.508 

10.503 

3.000 

1.000 

.222 

TABLES. 

REPEATED  MEASURES  MANOVA:  MISSION  AND 

FREEZE  POINT 

DEPENDENT  MEASURE:  SWAT 
OSO 


Effect 

Value 

F 

Hypothesis 

df 

Error 

df 

Sig. 

Eta  Squared 

MSN 

Pillai's  Trace 

.877 

21.447 

1.000 

3.000 

.019 

.877 

Wilks'  Lambda 

.123 

21.447 

1.000 

3.000 

.019 

.877 

Hotelling's  Trace 

7.149 

21.447 

1.000 

3.000 

.019 

.877 

Roy’s  Largest 

Root 

7.149 

21.447 

1.000 

3.000 

.019 

.877 

FRZPT 

Pillai's  Trace 

1.000 

4401.363 

3.000 

1.000 

.011 

1.000 

Wilks'  Lambda 

.000 

4401.363 

3.000 

1.000 

.011 

1.000 

Hotelling's  Trace 

13204.0 

90 

4401.363 

3.000 

1.000 

.011 

1.000 

Roy's  Largest 

Root 

13204.0 

90 

4401.363 

3.000 

1.000 

.011 

1.000 

MSN  * 
FRZPT 

Pillai's  Trace 

.920 

3.830 

3.000 

1.000 

.355 

.920 

Wilks' Lambda 

.080 

3.830 

3.000 

1.000 

.355 

.920 

Hotelling's  Trace 

11.489 

3.830 

3.000 

1.000 

.355 

.920 

Roy's  Largest 

Root 

11.489 

3.830 

3.000 

1.000 

.355 

.920 

95 


TABLE  6.  REPEATED  MEASURES  MANOVA  -  MISSION  AND  FREEZE  POINT 
DEPENDENT  MEASURE:  SWAT 


Effect 

DSO 

Value 

F 

Hypothesis  df 

Error  df 

Sig. 

MSN 

Pillai's  Trace 

.074 

.239 

1.000 

3.000 

.658 

Wilks'  Lambda 

.926 

.239 

1.000 

3.000 

.658 

Hotelling's  Trace 

.080 

.239 

1.000 

3.000 

.658 

Roy's  Largest  Root 

.080 

.239 

1.000 

3.000 

.658 

FRZPT 

Pillai's  Trace 

.892 

2.758 

3.000 

1.000 

.410 

Wilks'  Lambda 

.108 

2.758 

3.000 

1.000 

.410 

Hotelling's  Trace 

8.273 

2.758 

3.000 

1.000 

.410 

Roy's  Largest  Root 

■  8.273 

2.758 

3.000 

1.000 

.410 

MSN  *  FRZPT 

Pillai's  Trace 

.990 

33.226 

3.000 

1.000 

.127 

Wilks' Lambda 

.010 

33.226 

3.000 

1.000 

.127 

Hotelling's  Trace 

99.677 

33.226 

3.000 

1.000 

.127 

Roy's  Largest  Root 

99.677 

33.226 

3.000 

1.000 

.127 
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